Apparently our office manager ordered up a Jura machine today! Not sure the exact model but they usually just yolo and go for the top tier of stuff. It’s one of the z10 models. Since it plugs into the wall, I’m in IT and I like coffee, I’m sure I’ll have free rein and some expectation to manage the thing.
Any tips?
Our devs keep pushing boundaries on external LLM tooling. I personally don’t do anything complex enough to go past a 35b model on my MacBook, they do though when researching and debugging.
I know we won’t hit Claude level or other cloud llm performance, but I think we can really mitigate a lot of their usage with something in prem. Let’s say that reasonable money is no object here, but under 100k. No redundancy is required. 35 devs.
What would you spec for hardware? How big of a model if a degree of compromise is acceptable? How would configure it from a user perspective?