I am trying to build a budget PC. I have most of the parts now, and only the case is pending. My main goal is to get a budget PC case. I am also looking to support future CPU or GPU upgrades, so the case should be able to handle that. I don’t have any size preference. So far, I have checked the Lian Li A3 and the O11D.
Any suggestions?
For example, if I have 128 GB of system RAM but only 16 GB of VRAM, am I still limited to models that fit within GPU memory (aside from CPU offloading techniques like MoE)? Are there ways to increase context size using system ram with usable token generation speed?
I have spare gpu , ram and psu and trying to build a small form factor machine for basement. Any suggestions on combo and case?
I think we all agree that 5090 or 3090 is the best local gpu setup for local llm. But what about cpu? What's the best and cheap cpu for this setup?
I am trying to understand the best practices for monitoring and Observability in fastAPI. Does it come with different metrics and otel out of the box? Also, how are you using other tools and library along with it to make it production ready?