The launch
On October 8–9, 2026, Underdog, the on-device assistant from Conway Research, released Saluki 27B under Apache 2.0. It's a 2-bit compressed GGUF version of Qwen3.8-27B that takes 7.89 GB, versus 54 GB for the full BF16 model. It runs in stock llama.cpp, no forks.
The interesting detail: the compression was tuned to protect tool calling, the skill that turns a chat model into an agent.
The numbers (published by Underdog)
- 96% average retention across 9 benchmarks versus full Qwen3.8-27B.
- Underdog Bench (120 BFCL v4 tasks, frozen before testing): Saluki 88, full model 84.
- Parallel tool calls (100 BFCL v4 tasks): Saluki 42, full 35.
- IFEval (instruction following): 93.5 versus 91.5.
In other words, on tool use and instruction following, the compressed version matches or beats the original.
Where it loses
Compression takes its toll on heavy reasoning:
- AIME 2025: 79.2 versus 96.7.
- MuSR: 67.5 versus 79.6.
- SWE-bench Verified (50 issues): fixes 30 versus 33 for the full model.
Math and reasoning land at about 82% to 85% of the original. A result above the original after compression also deserves caution: it may reflect fine-tuning focused on those tasks and normal test variance. All numbers come from the vendor.
Why it matters
A 27B model with good tool use used to need a server GPU. At under 8 GB, it fits on a consumer card or a laptop with unified memory. That opens room for local agents: task automation, queries to internal systems, and assistants that don't send data outside.
The technical lesson holds too: compression doesn't have to be neutral. You can choose which skills to protect depending on the use case.
How to evaluate it
- Test with your own tools and function schemas, not just benchmarks.
- For math or long reasoning, keep a larger model as a fallback.
- Check the base model's (Qwen) license in addition to Saluki's Apache 2.0.
- Measure memory and speed on your real hardware.
Sources
- MarkTechPost (Oct 9, 2026): https://www.marktechpost.com/2026/10/09/meet-the-underdog-saluki-27b-a-2-bit-qwen3-8-27b-that-beats-the-original-at-tool-calling/
- Hugging Face, ConwayResearch/Underdog-Saluki-27B-1.0: https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
Want to run local AI agents without sending data to the cloud? KGA IT offers a free consultation: kga-it.com.