EdgeRazor is a lightweight framework for edge AI, designed to train models that are smaller, faster, and deployable across diverse hardware, ranging from mobile and edge endpoints to latency-sensitive clouds. The EdgeRazor framework seamlessly integrates model compression techniques into existing full-precision training pipelines with minimal code modification, preserving promising task performance and enabling low-cost and high-efficiency computations.

EdgeRazor currently focuses on low-bit LLM compression via configurable quantization-aware distillation. In terms of quantization, EdgeRazor supports quantizing weights (including embedding and lm_head layers), activations, and KV cache. Quantized bit-widths include the uniform 1.58-bit and 4-bit, as well as matrix-wise mixed-precision, such as 2.79-bit (50% 4-bit + 50% 1.58-bit) and 1.88-bit (12.5% 4-bit + 87.5% 1.58-bit). In terms of distillation, EdgeRazor offers the logits, features, and attention distillation, all of which can be flexibly combined within a unified configuration interface.

EdgeRazor achieves the state-of-the-art performance across a range of models, including base LLMs, instruction-tuned LLMs, and multimodal LLMs. For W-A8-KV8 quantization, Qwen3-0.6B-EdgeRazor attains average scores of 47.80 / 44.10 / 41.76 / 39.81 at 4-bit / 2.79-bit / 1.88-bit / 1.58-bit, corresponding to compression ratios of 3.94× / 5.05× / 6.40× / 7.03×, respectively. In comparison, the best prior methods achieve 45.74 / 37.38 / 30.49 at 4-bit / 3-bit / 2-bit with compression ratios of 2.21× / 2.47× / 2.78×.

Figure: The EdgeRazor framework with lightweight model training pipeline.

• 🔥 [2026-04]: 📄 Paper-EdgeRazor is available on arXiv:2605.04062 and Hugging Face Paper!
• 🔥 [2026-04]: 🚀 EdgeRazor Playground is launched and open-sourced! CPU-friendly! Have a try!
• 🔥 [2026-04]: 🏅 CACC 2025 Final (China Algorithm Capability Competition) apply EdgeRazor as a solution in the AI subject!
• 🔥 [2026-04]: 🏆 Low-bit LLMs by EdgeRazor is released! Check our Hugging Face collection: zhangsq-nju/edgerazor-nbit.
• 🔥 [2026-04]: 🛠️ Open-sourced EdgeRazor-V1 is released! Now configurable on diverse models for seamless integration and customization!
• 🔥 [2025-10]: 📄 Paper-TernaryCLIP is available on arXiv:2510.21879!

• News
• Contents
• Quick Start
  • Installation
  • Usage
  • Docker
  • Playground
• Main Techniques
• Applications
• Model Zoo
  • LLMs
  • MLLMs
• Todo List
• Acknowledgements
• Citation
• Contributor List

• Download from PyPi

• Download from GitHub (latest version)

After installation, you can integrate EdgeRazor into your existing training pipeline to build lightweight models.

1. Use unified configuration by yaml, json or dict.

2. Seamlessly integrate EdgeRazor into your FULL-PRECISION model training pipeline!

3. Below are LLM examples of both the high-level and low-level API usage

Lightweight models are available from checkpoints trained with EdgeRazor. For example, you can convert Qwen3-EdgeRazor-4bit checkpoints to Q4_0 GGUF models. We also provide ready-to-use quantized models in our collection, including Qwen3-0.6B-EdgeRazor-GGUF and Qwen3-1.7B-EdgeRazor-GGUF.

EdgeRazor Playgound is CPU-friendly! Enjoy low-bit LLMs from EdgeRazor on your edge devices!

Quantization-Aware Distillation (QAD):

• Configurable mixed-precision quantization for weights
• Configurable knowledge distillation pipelines between 16-bit and $n$-bit models
• Knowledge distillation methods: Adaptive Feature Distillation (AFD), Entropy-Aware KL Divergence (EAKLD)

Figure: Workflow of the EdgeRazor framework.

• Lightweight ViT-S/16 for MNIST, check here.
• Lightweight ViT-S/16 for RetinaMNIST, check here.
• Lightweight ResNet-18, check here.
• Lightweight Qwen3-0.6B/1.7B, check here.
• Lightweight MobileLLM-ParetoQ-350M-BF16, check here.
• Lightweight Qwen2.5-Omni-7B, check here.

• Average Performance (Avg.): average of performance scores in multiple tasks using lm-eval v0.4.9.1 with tasks.

  • Tasks for instruct LLMs: arc_easy, arc_challenge, hellaswag, boolq, social_iqa, openbookqa, piqa, winogrande, hendrycks_ethics, truthfulqa_mc2, mmlu, gsm8k, humaneval_instruct, ifeval.
  • Tasks for base LLMs: arc_easy, arc_challenge, hellaswag, boolq, social_iqa, openbookqa, piqa, winogrande, hendrycks_ethics, truthfulqa_mc2, mmlu, gsm8k, humaneval.
  • Except for 5-shot gsm8k, all other tasks are 0-shot.

• Hub Link: We provide the original quantized checkpoints. We also transfer the checkpoints into GGUF (llama.cpp) and GPTQ (GPTQModel, working in progress) formats if compatible.

• Video-MME and MLVU are video understanding tasks using lmms-eval v0.5.0 with tasks.

EdgeRazor is continuously evolving! Here's what's coming:

• Lightweight MobileLLM, Qwen3, and Qwen2.5-Omni: training code
• New API: EdgeRazorCausalLMTrainer for compressing text LLMs
• Upgrade to support the newest dependencies

Have ideas or suggestions? We welcome and appreciate any contributions and collaborations! Please feel free to submit issues or pull requests! 🚀

The deployment demos utilize llama.cpp and ChatterUI.

If you find our papar and code useful in your research, please consider kindly citing our papers ✏️:

This project was supported by LAMDA and Assistant Professor Shao-Qun Zhang. Shu-Hao Zhang is the core developer and maintainer of EdgeRazor-V1. Xiang-Sheng Deng and Le-Tong Huang jointly participated in the development of this project.