What is Quantization-Aware Distillation?
Quantization-Aware Distillation (QAD) is a technique that enables developers to run large language models at high speeds and low memory footprints without sacrificing quality. The QAD checkpoints for LFM2.5 models have been released, allowing for 97% recovery of average accuracy lost to quantization. This breakthrough has significant implications for edge deployment and real-world applications.


Quantization-Aware Distillation (QAD) is a total game-changer for developers working with large language models - it's like having your cake and eating it too. By distilling a high-precision teacher model into a quantized student model, QAD lets you create models that run at crazy-high speeds and have super-low memory footprints, all without the usual quality drop. And with the recent release of QAD checkpoints for LFM2.5 models, developers can now tap into the power of these models in real-world applications, which is a major milestone.
The QAD checkpoints have been put to the test against post-training quantization (PTQ) checkpoints, and the results are pretty impressive. Across all four LFM2.5 models, the QAD checkpoints retain a whopping 97% of their respective BF16 baseline performance - that's huge. This means developers can now deploy these models on edge devices like smartphones and Raspberry Pi without sacrificing quality, which is a big deal. And get this - the QAD checkpoints are also on par with external post-training quantization checkpoints, like Unsloth's UD-Q4_K_XL.
In terms of speed and size, the QAD checkpoints are just as impressive. On real edge hardware like MacBook Pro and Samsung Galaxy S26 Ultra, the QAD checkpoints match Q5_K_M quality within evaluation variance at a 4-33% higher decode throughput - that's a significant boost. This means developers can now build apps that are not only faster but also more accurate, which is a win-win. And the best part? The QAD checkpoints are available on Hugging Face, so developers can get started with them using the llama.cpp runtime or any other runtime that supports GGUF Q4_0 artifacts.
The implications of QAD are huge - it enables the deployment of large language models in real-world applications where speed and memory footprint are critical. With QAD, developers can now build apps that are faster, more accurate, and more efficient, opening up new possibilities for edge deployment and beyond. Whether you're building a chatbot, a virtual assistant, or a language translation app, QAD is a technique that can help you achieve your goals - it's definitely worth checking out.
Source: Hugging Face
Related posts

Reson8 Releases Rumble-2-Flash 0.5B Speech Model for 40 Languages
Reson8 released Rumble-2-Flash, a 0.5-billion-parameter speech recognition model covering 40+ languages. Launched November 18, 2026 on Hugging Face under Apache 2.0, it runs in standard Transformers and vLLM without custom code. Distilled from the larger Rumble-2 via the GOLD on-policy method, it scores 6.96% error on the multilingual Open ASR Leaderboard, beating Meta's 7B omniASR-LLM and Whisper large-v3, and processes an hour of audio in about half a second on a single GPU.

Osmo Claims Its AI Model Matches Human Smell Ability
Osmo, a Google Brain spinout, claims its AI model achieves "human-level olfactory intelligence," detecting and discriminating smells like an average person and creating fragrances comparable to a junior perfumer. Founder Alex Wiltschko says the tech, trained on over ten million human scent labels, is already used by major fragrance firms and identified repellents stronger than DEET. Critics argue marketing claims lack documentation and the system struggles with contextual smell interpr

Google Launches Free Tasks AI Agent Replacing Spark
Google has rebranded its Spark AI agent as Tasks and made it free for users in the US, Canada, Japan, Korea, Australia, and India. Powered by Gemini 4 Hydrogen, the agent handles planning chores like meal plans and calendar organization. US users gain additional capabilities, including calling businesses, making purchases via Google Pay, and delegating tasks via email or SMS. Each session runs in an isolated virtual machine, pausing for confirmation before sensitive actions.