r/unsloth

For agent models, PinchBench and Tau2 may matter more than one more AIME headline。

I still think AIME and GPQA matter. They say something real about capability ceilings. For agent models, though, I reach first for execution-heavy, tool-heavy, multi-step signals. That is why Ring-2.6-1T caught my eye: PinchBench: 87.60, Tau2-Bench Telecom: 95.32, and ClawEval: 63.82 sit alongside AIME 26: 95.83, GPQA Diamond: 88.27, and ARC-AGI-V2: 66.18. For production-style agents, I care first about whether the model can keep a workflow moving, coordinate tools cleanly, and avoid spending deep reasoning on every intermediate step. The public high / xhigh framing fits that story too, with deeper reasoning available when you need it instead of dominating every path.

unsloth studio를 업데이트 한 이후부터 프롬프트 입력에서 한글 입력이 전혀 안된다.
운영 환경은 mac mini

unsloth studio의 버그야? 아니면 한글 같은 다국어 입력 문제 해결 방법이 있나??

The Unsloth team has been truly amazing in breadth and depth of releases. I’m super excited to try the 27b in particular.

The Qwen3 CoderNext model has actually surprised me in functionality when thinking is less valuable, like feeding Aider.

I would be grateful if this got MTP turned on with Unsloth’s high quality quants!!

Anyone else a fan?

I finally got around to installing Unsloth Studio and am giving it a whirl on my Windows machine. I have an RTX6k and am trying to load the latest QWEN MTP model you guys created. However no matter what I do, all models are loading in RAM instead of the GPU.

From what i see online it means there's a cuda driver mismatch most likely. But i installed 12.9 and updated all my environment variables. So I'm not sure why this is still happening.

Any thoughts or help?

Got a new laptop Lenovo slim 7i (32gb ram)
was thinking if it can run this

Prompt - Write a 500 word explanation of Rust ownership with examples.

[ Prompt: 13.5 t/s | Generation: 6.7 t/s ]

Hi i have looping issue when i try the new MTP branch version of llama.cpp
My config:

[*]

chat-template-kwargs = {"preserve_thinking":true}

reasoning-budget = 4096

reasoning-budget-message = "Reasoning budget reached. Conclude the analysis and provide the final answer."

device = Vulkan1

gpu-layers = all

no-mmproj-offload = 1

batch-size = 2048

ctx-size = 128000

ubatch-size = 512

temp = 0.6

top-p = 0.95

top-k = 20

min-p = 0.00

presence-penalty=0.0

repeat-penalty=1.0

cache-prompt = 1

timeout = 600

reasoning = on

image-min-tokens = 1024

metrics = 1

fit-target = 0

no-mmap = 1

jinja = 1

prio = 3

reasoning = on

no-warmup = 1

parallel = 1

flash-attn = on

port = 8001

threads = 16

threads-batch = 16

cache-type-k = q8_0

cache-type-v = q8_0

kv-unified = true

ctx-checkpoints = 64

checkpoint-every-n-tokens = 2048

cache-ram = 20480

mlock = 1

main-gpu = 1

verbose=1

[Qwen3.6-27B-MTP-UD-Q6_K]

model = C:\Users\user\.cache\huggingface\hub\models--unsloth--Qwen3.6-27B-MTP-GGUF\snapshots\53b097416d6346f849b530e4bc1b5590dfe9d758\Qwen3.6-27B-Q6_K.gguf

mmproj = C:\Users\user\.cache\huggingface\hub\models--unsloth--Qwen3.6-27B-MTP-GGUF\snapshots\53b097416d6346f849b530e4bc1b5590dfe9d758\mmproj-BF16.gguf

cache-type-k = q4_1

cache-type-v = q4_1

spec-type = draft-mtp

spec-draft-n-max = 2

---------

i can see in terminal the LLM looping

[53923] srv update_slots: run slots completed [53923] que start_loop: waiting for new tasks [53923] que start_loop: processing new tasks [53923] que start_loop: processing task, id = 1798 [53923] que start_loop: update slots [53923] srv update_slots: posting NEXT_RESPONSE [53923] que post: new task, id = 1799, front = 0 [53923] slot get_n_draft_: id 0 | task 0 | max possible draft: 15217 [53923] slot update_batch: id 0 | task 0 | generate_draft: id=4013, #tokens=20320, #draft=1, pos_next=20320 [53923] srv update_slots: decoding batch, n_tokens = 2 [53923] set_adapters_lora: adapters = 0000000000000000 [53923] adapters_lora_are_same: adapters = 0000000000000000 [53923] set_embeddings: value = 1 [53923] slot update_slots: id 0 | task 0 | restoring speculative checkpoint (pos_min = 20319, pos_max = 20319, size = 748) [53923]

srv update_slots: run slots completed [53923] que start_loop: waiting for new tasks [53923] que start_loop: processing new tasks [53923] que start_loop: processing task, id = 1799 [53923] que start_loop: update slots [53923] srv update_slots: posting NEXT_RESPONSE [53923] que post: new task, id = 1800, front = 0 [53923] slot get_n_draft_: id 0 | task 0 | max possible draft: 15217 [53923] slot update_batch: id 0 | task 0 | generate_draft: id=4013, #tokens=20320, #draft=1, pos_next=20320 [53923] srv update_slots: decoding batch, n_tokens = 2 [53923] set_adapters_lora: adapters = 0000000000000000 [53923] adapters_lora_are_same: adapters = 0000000000000000 [53923] set_embeddings: value = 1 [53923] slot update_slots: id 0 | task 0 | restoring speculative checkpoint (pos_min = 20319, pos_max = 20319, size = 748) [53923]

srv update_slots: run slots completed [53923] que start_loop: waiting for new tasks [53923] que start_loop: processing new tasks [53923] que start_loop: processing task, id = 1800 [53923] que start_loop: update slots [53923] srv update_slots: posting NEXT_RESPONSE [53923] que post: new task, id = 1801, front = 0 [53923] slot get_n_draft_: id 0 | task 0 | max possible draft: 15217 [53923] slot update_batch: id 0 | task 0 | generate_draft: id=4013, #tokens=20320, #draft=1, pos_next=20320 [53923] srv update_slots: decoding batch, n_tokens = 2 [53923] set_adapters_lora: adapters = 0000000000000000 [53923] adapters_lora_are_same: adapters = 0000000000000000 [53923] set_embeddings: value = 1 [53923] slot update_slots: id 0 | task 0 | restoring speculative checkpoint (pos_min = 20319, pos_max = 20319, size = 748) [53923]

----

Does somebody also has this issue, better yet, does have somebody solution? This loops until timeout

Hey everyone,

I’m a 22-year-old independent researcher, and I’ve been trying to tackle the "Memory Wall" for long-context LLMs. Standard methods either quantize precision (which hits a hard limit) or use token eviction (which degrades reasoning).

I just published an open research draft for a different geometric approach called HoloKV.

The concept: Instead of appending new memory slots, HoloKV multiplexes (stacks) k tokens into a single physical memory slot. It uses deterministic +1/-1 orthogonal phase keys (inspired by CDMA telecommunications) to separate the signals.

To make it work natively with modern architectures, I introduced:

1. Variance Normalization: A sqrt(k) penalty to prevent Softmax entropy collapse caused by superimposing vectors.
2. Strict Even-Boundary Rule: A constraint on phase-key generation that perfectly preserves the 2D rotary commutative math of RoPE (Llama/Qwen).
3. LoRA Denoising: Injecting Query/Value LoRA adapters via Knowledge Distillation to natively filter out the Gaussian background static.

The Ask:
I have successfully built the mathematical simulator in PyTorch to prove the orthogonal extraction and RoPE preservation work. However, I am a solo dev working on a GTX 1650. To actually realize the 75%+ physical VRAM savings, this needs a custom SRAM Active Accumulation Buffer written in OpenAI Triton or CUDA to prevent the "Read-Modify-Write" penalty.

I am open-sourcing the math and the paper. If there are any Triton/FlashAttention kernel engineers here who want to collaborate and help me build the hardware kernel, please reach out or open a PR!

**Paper & Code:**https://github.com/0sami0/HoloKV