▲ 5 r/LocalLLaMA
How difficult is distilling?
I remember a year or so ago when DeepSeek R1 came out and it was pretty quickly distilled into Llama 3 8b and Qwen 2.5 (?) 7b. Why don’t we see more distilled models? How expensive is it? How many tokens or prompts does it take?
u/GreedyWorking1499 — 6 days ago