I think this issue is fixed in this release.

From what I’ve seen, it’s definitely worth quantizing. I’ve used llama 3 8B (fp16) and llama 3 70B (q2_XS). The 70B version was way better, even with this quantization and it fits perfectly in 24 GB of VRAM. There’s also this comparison showing the quantization option and their benchmark scores:

Source

To run this particular model though, you would need about 45GB of RAM just for the q2_K quant according to Ollama. I think I could run this with my GPU and offload the rest of the layers to the CPU, but the performance wouldn’t be that great(e.g. less than 1 t/s).

Yes, you can find it here.

Are you using mistral 7B?

I also really like that model and their fine-tunes. If licensing is a concern, it’s definitely a great choice.

Mistral also has a new model, Mistral Nemo. I haven’t tried it myself, but I heard it’s quite good. It’s also licensed under Apache 2.0 as far as I know.

Currently, I only have a free account there. I tried Hydroxide first, and I had no problem logging in. I was also able to fetch some emails. I will try hydroxide-push as well later.

I haven’t heard of Hydroxide before; thank you for highlighting it! Just one question: Does it also require a premium account like the official bridge, or is it also available for free accounts?

This reminds me of one of my older projects. I wanted to learn more about network communications, so I started working on a simple P2P chat app. It wasn’t anything fancy, but I really enjoyed working on it. One challenge I faced was that, at the time, I didn’t know how to listen for user input while handling network communication simultaneously. So, after I had managed to get multiple TCP sockets working on one thread, I thought, why not open another socket for HTTP communication? That way, I could incorporate a fancy web UI instead of just a CLI interface.

So, I wrote a simple HTTP server, which, in hindsight, might not have been necessary.

I discovered a channel called ‘Just Alex’ awhile ago, and I’ve binge-watched most of his videos. He makes videos about a variety of activities he engages in. For example, two years ago, he started beekeeping as a hobby. In these videos, he shows the progress he has made since then, taking care of his bees and harvesting their honey.

He also has videos on collecting mushrooms, growing crops, and traveling around Europe.

What I like about him is how calming his videos are and the amount of effort he puts into making them interesting, without resorting to clickbait content.

You are right. I think it’s time to release it on Google play. I also want to focus on getting the app into the main F-Droid repo as well.