Compare model versions

Diff

Before

Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b10665">b10665</a> for quantization.

Original model: https://huggingface.co/Qwen/Qwen3.8-Flash-Next

**Model details:**
- Parameter count: 180B
- Input support: text, image (with mmproj file) - [details](#multimodal)
- Speculative decoding: no
- imatrix: yes - [details](#imatrix)

[How to run](#how-to-run)

After

Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b10665">b10665</a> for quantization.

Original model: https://huggingface.co/Qwen/Qwen3.8-Flash-Next

**Model details:**
- Parameter count: 180B
- Input support: text, image (with mmproj file) - [details](#multimodal)
- Speculative decoding: yes (MTP draft model) - [details](#speculative-decoding)
- imatrix: yes - [details](#imatrix)

[How to run](#how-to-run)

Diff

Before

These quants run with [llama.cpp](https://github.com/ggml-org/llama.cpp) - installable in one line via [llama.app](https://llama.app/):

```
curl -LsSf https://llama.app/install.sh | sh
llama-server -hf bartowski/Qwen3.8-Flash-Next-GGUF:Q4_K_M
```

llama-server includes a built-in chat web UI, served at http://localhost:8080 by default.

These quants were made with llama.cpp release b10665 - if this model's architecture is newly supported, you'll need that release or newer to run them.

They also work in: [LM Studio](https://lmstudio.ai/) · [koboldcpp](https://github.com/LostRuins/koboldcpp) · [ramalama](https://github.com/containers/ramalama) · [Jan AI](https://www.jan.ai/) · [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui) · [LoLLMs](https://github.com/ParisNeo/lollms) · [Atomic Chat](https://atomic.chat/)

After

These quants run with [llama.cpp](https://github.com/ggml-org/llama.cpp) - installable in one line via [llama.app](https://llama.app/):

```
curl -LsSf https://llama.app/install.sh | sh
llama-server -hf bartowski/Qwen3.8-Flash-Next-GGUF:Q4_K_M
```

llama-server includes a built-in chat web UI, served at http://localhost:8080 by default.

These quants were made with llama.cpp release b10665 - if this model's architecture is newly supported, you'll need that release or newer to run them.

The `mtp-` draft files were made with release b11564 and need that release or newer.

They also work in: [LM Studio](https://lmstudio.ai/) · [koboldcpp](https://github.com/LostRuins/koboldcpp) · [ramalama](https://github.com/containers/ramalama) · [Jan AI](https://www.jan.ai/) · [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui) · [LoLLMs](https://github.com/ParisNeo/lollms) · [Atomic Chat](https://atomic.chat/)

Diff

Before

Not present in the earlier snapshot.

After

This repo includes an MTP draft model, extracted from the Multi-Token Prediction layer in [Qwen/Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)'s checkpoint and shipped here as separate `mtp-` prefixed files: [mtp-Qwen3.8-Flash-Next-bf16.gguf](https://huggingface.co/bartowski/Qwen3.8-Flash-Next-GGUF/blob/main/mtp-Qwen3.8-Flash-Next-bf16.gguf), [mtp-Qwen3.8-Flash-Next-Q8_0.gguf](https://huggingface.co/bartowski/Qwen3.8-Flash-Next-GGUF/blob/main/mtp-Qwen3.8-Flash-Next-Q8_0.gguf), [mtp-Qwen3.8-Flash-Next-Q4_0.gguf](https://huggingface.co/bartowski/Qwen3.8-Flash-Next-GGUF/blob/main/mtp-Qwen3.8-Flash-Next-Q4_0.gguf).

The MTP layer is not inside the quants above - these files were made before llama.cpp could convert it for this architecture, and it has since been extracted as a draft model on its own. Nothing loads it unless you ask: add `--spec-type draft-mtp` to the llama-server command shown above, and llama.cpp finds the draft in this repo by its filename and downloads it alongside the quant:

```
llama-server -hf bartowski/Qwen3.8-Flash-Next-GGUF:Q4_K_M --spec-type draft-mtp
```

If you're loading files manually, download one of the `mtp-` files and pass it with `-md` - llama.cpp detects the speculative decoding type from the file's metadata, so no extra flags are needed:

```
-md mtp-Qwen3.8-Flash-Next-Q8_0.gguf
```

The draft files need llama.cpp release b11564 or newer, which is when this architecture's MTP export landed. The draft is quantized statically (no imatrix); it only proposes tokens the main model verifies, so its precision affects speed, not output quality.