Heya everyone,
So I'm having a bout of buyers remorse.
Somewhat on impulse, I purchased 2 x modded 22gb 2080ti's from alibaba.
In my head I was gonna host a model to run openclaw, something in around the 20-30b parameter count. I dont mind if it's a bit slower to do things, but the gpu's had the memory capacity for it. I also bought an nvlink to go between them, and I'm using vllm rather than ollama to take advantage of the tensor parallelism.
Yup, I'm aware these are likely former mining cards etc, but at the time it seemed like a good idea.
So, what would be a good model to use for light coding work (heavy coding will be formed out to codex oauth), homelab maintenance, RAG, email triage and document creation?
Alternatively, am I being delusional?
Keen for people's thoughts and experiences.
Cheers!