u/big_black_cucumber

Hey guys!

I have a project in mind where I want to use a local hosted llm for.

However, I want my compute power to be minimal. So i was basically wondering if any of you had also already tried something like this out?

I want find the best model to host on my raspberry pi5 8GB for basic text generation with a decent context window.

All suggestions are much appreciated!