small language models

Auto Added by WPeMatico

Post-training Quantization for Edge AI

ai, AI (Artificial Intelligence), Artificial Intelligence, AWQ, edge ai, GGUF, GPTQ, INT4 quantization, INT8 quantization, LLM quantization, model compression, on-device inference, post-training quantization, small language models, SmoothQuant, Uncategorized

Why it matters: Post-training quantization for edge AI: how INT8 and INT4 shrink models 4-16x, what accuracy costs, and how to deploy fast with GPTQ, AWQ, and GGUF.

Post-training Quantization for Edge AI Read More »

Mixture of Experts Small Models

active parameters, ai, AI (Artificial Intelligence), Artificial Intelligence, deepseek, LLM architecture, Mixtral, mixture of experts, model efficiency, MoE models, On-Device AI, Qwen3, routing, small language models, sparse activation, Uncategorized

Why it matters: Mixture of experts small models explained: how sparse activation, active versus total parameters, and routing deliver capable AI at low compute cost.

Mixture of Experts Small Models Read More »