4-bit Qwen3.6 MTP GGUF managed to search 70+ sites from a single prompt.

Try this locally with Unsloth Studio on 20GB RAM.

Unsloth now supports automatic MTP + speculative decoding for supported models. Unsloth also now auto-selects the best MTP settings for your specific device (Mac, CPU, GPU etc.)

We also fixed many bugs and issues including tokens/s not showing up correctly and MTP not being applied properly.

GitHub: https://github.com/unslothai/unsloth

Hey guys, Qwen3.6 can run ~1.4–2.2× faster with no accuracy change due to MTP. You can run this locally on just 18GB RAM, VRAM or unified memory.

The Qwen3.6 Unsloth GGUFs are now out of experimental mode, llama.cpp has merged many PRs, and MTP is now properly supported in Unsloth. MTP is now ready!

Please use the latest Unsloth `v0.1.41-beta`, not `v0.1.405-beta` which is older. In Studio, we automatically set all the params for you depending on your specific hardware so you get the near best results (you can still change it)

Qwen3.6-27B MTP can run at 160 tokens/s. Qwen3.6-35B-A3B MTP GGUF reaches 240 tokens/s. We also uploaded MTP GGUFs for Qwen3.5!

27B MTP GGUF: https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF
35B-A3B MTP GGUF: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF

Guide: https://unsloth.ai/docs/models/qwen3.6#mtp-guide

Thank you! We've got lots of releases this week as well.

Hey guys, Qwen3.6 MTP GGUFs now work in Unsloth Studio: https://github.com/unslothai/unsloth

Just update Unsloth Studio or do a fresh install.

MacOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

irm https://unsloth.ai/install.ps1 | iex

As always huge thanks to llama.cpp and devs for making this possible.

We'll be doing a new pypi release with lots of new updates tomorrow! Lots!!!

Hey guys, some of you may seen our Qwen3.6 MTP GGUFs. MTP (Multi Token Prediction) speculative decoding enables models like Qwen3.6 to have ~1.4-2x faster generation with no change in accuracy. This enables Qwen3.6 27B and 35B-A3B to have >1.4x speed-up over the original baseline which is especially useful for local models.

Qwen3.6 27B can now do 140 tokens / s generation and Qwen3.6 35B-A3B 220 tokens / s generation! See MTP Benchmarks for more details.

Regarding draft tokens, we found 2 to be the best. The acceptance rate defs drops, so it's probs best in general to stick with 2. For coding, maybe 3 will work fine since more tokens probs gets accepted

You must use the specific llama.cpp PR branch which we give instructions for in our guide below. Unsloth Studio will support it once the PR is merged.

• Guide + breakdown + benchmarks: https://unsloth.ai/docs/models/qwen3.6#mtp-guide
• Qwen3.6-27B MTP: https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF
• Qwen3.6-35B-A3B MTP: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF

We're now uploading MTP quants for Qwen3.5 smaller models. Thank you!

Hey guys, we're super excited to announce that Unsloth has officially joined the PyTorch Ecosystem! 🔥🦥

In case you didn't know, Unsloth is an open-source project that makes training & running models more accurate and faster with less compute. Our mission is to make local AI accessible to everyone. Unsloth will remain as an independent open-source project, separate from the PyTorch Foundation.

Blog: https://unsloth.ai/blog/pytorch

GitHub: https://github.com/unslothai/unsloth

Thanks to all of you for making this possible! 💕

Hey guys, we did bug fixes for Unsloth where chat history was not being shown (existing chat history is not lost) and attachments not attaching correctly. It was a visual bug and render-only. So please update to the latest version of Unsloth: https://unsloth.ai/docs/new/studio/install

Latest version: v0.1.39-beta

Use 2026.5.2 or directly call curl -fsSL https://unsloth.ai/install.sh | sh or unsloth studio update to update

Thanks so much!

Hey guys, you can now run open LLMs in Claude Code, Codex and OpenClaw via Unsloth's API inference endpoint and we made lots of tutorials for it!

Use Gemma 4 and Qwen3.6 GGUFs for local agentic coding on 24GB RAM.

Run with self-healing tool calls, code execution, web search via the Unsloth API endpoint and llama.cpp.

Guide: https://unsloth.ai/docs/basics/api

Unsloth makes it easy to deploy a fast API inference endpoint that provides:

• Self-healing tool calling, which helps reduce broken or malformed tool calls by 50%
• Code execution support, allowing Bash and Python execution for more accurate code outputs.
• Advanced Web search that visits and actually reads webpages to gather in-depth info.
• Automatic inference settings for GGUF models (temp, top-k etc.)

Please update Unsloth to leverage this new update and let us know if you have any feedback. Thank you!!

Hey guys we recently released new NVFP4 quants for Qwen3.6 which you can run via vLLM, SGLang etc for fast inference on supported GPUs like RTX 50 series.

The NVFP4 quants ate calibrated on the Hugging Face UltraChat dataset with sequences up to 16K context length and an approximately 2M-token calibration budget.

27b: https://huggingface.co/unsloth/Qwen3.6-27B-NVFP4

35b-a3b: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4

Please note they may be improvements in the near future for NVFP4 quants. Let us know if you have any suggestions thanks. 🙏

Hey guys, we worked with Mistral to fix Mistral Medium 3.5 inference affecting some implementations, and released updated GGUFs with the fix (NOT related to Unsloth or quants). Mistral 3.5 now works properly in transformers AND llama.cpp.

The issue was caused by a YaRN parsing quirk affecting some implementations. Changing mscale_all_dim from 1 to 0 resolved it. We also fixed mmproj files generation.

Mistral has pushed our fixes to their official repo. The YaRN scaling multiplier is applied correctly, fixing forgetting previous conversations.

Thanks a lot to everyone who firstly reported the issue to us after using GGUFs.

Guide: https://unsloth.ai/docs/models/mistral-3.5

GGUFs: https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF

Have a lovely weekend guys!

Hey guys IBM released their new Granite-4.1 series of models. They're all dense and you can run the 3B model on just 3GB of RAM.

Guide: https://unsloth.ai/docs/models/ibm-granite-4.1

GGUFs:

• unsloth/granite-4.1-3b-GGUF
• unsloth/granite-4.1-8b-GGUF
• unsloth/granite-4.1-30b-GGUF

Thanks guys!

Hey guys, just letting you all know that Unsloth does NOT use Pytorch Lighting anywhere in the package thus, Unsloth is not affected. And you do not need to worry about the recent compromise. Thank you. 🙏

You can ask any questions here ornon GitHub: https://github.com/unslothai/unsloth/issues/5236

Mistral releases Mistral Medium 3.5, a new vision reasoning model. 🔥

Mistral-Medium-3.5-128B offers highly competitive performance for models 5x its size.

We’re working with Mistral on llama.cpp GGUF implementation. Testing shows that this behavior occurs regardless of who or how the model was converted GGUF. The model initially responds correctly, but over long context, does not work properly.

Mistral has now labeled GGUF support as a WIP (work in progress). The issue appears most likely to be with the current GGUF parser. Will update once resolved.

NVIDIA releases Nemotron-3-Nano-Omni, a new 30B open multimodal MoE model.

Nemotron-3-Nano-Omni-30B-A3B is the strongest omni model for its size and supports audio, video, image and text.

Run 4-bit on ~25GB RAM or 8-bit on 36GB.

GGUF: https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF

Guide: https://unsloth.ai/docs/models/nemotron-3-nano-omni

Hey guys just recently, Unsloth is now one of the top 10 most followed organizations on Hugging Face! 🤗🦥

Thanks so much as always for all the support and we couldn't have done this without you guys! :)

Our Hugging Face profile: https://huggingface.co/unsloth

DeepSeek releases DeepSeek-V4 their latest SOTA open models. There are two models:

• DeepSeek-V4-Pro: 1.6T params / 49B active
• DeepSeek-V4-Flash: 284B params / 13B active.
• DeepSeek-V4-Pro rivals Claude-Opus-4.6-Max, GPT-5.4-xHigh.
• They support 1M context length, thinking and set new records for Codeforces.

Tech Report: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf

Hugging Face: https://huggingface.co/collections/deepseek-ai/deepseek-v4

Hey guys we showcase the power of 2-bit Qwen3.6-27B and Unsloth Studio!

2-bit Qwen3.6-27B GGUF made 26 tool calls, triaged 15 GitHub issues, executed code, fixed, tested + reproed our repo’s 3 latest issues. 🔥

We now added a Preserve thinking toggle! P.S. give Unsloth studio a try or update it as we added maaaany new features and introduced a whole new look!

Try it yourself via Unsloth Studio: https://github.com/unslothai/unsloth