u/Glas_Gow

Hey, I hope everyone is well.

I would like to ask about what is the best approach in terms of model settings for relatively high RAM (64 GB), but low GPU (3050 Ti 4GB - Laptop) PCs? My CPU is Intel i7-11800H. GPU offload is practically difficult/impossible with such low amount of VRAM.

Any model over 35B is useless on my PC. Also, what quantisation do you suggest?

I am new into local LLMs and still learning. Helpful comments appreciated. Thanks in advance.