AI Skills

In recent times, the popularity of transformer-based LLMs and LLM applications such as AI agents has skyrocketed. Compute is in high demand, while models soar in parameter count—reaching hundreds of billions and trillions of parameters in the largest LLMs. Luckily, researchers have been moving towards techniques to reduce the compute and VRAM needed to store, train, and run models. This is where small language models (SLMs) come in. Small language models are neural language models that are much smaller in size (typically billions of parameters or fewer) than today’s massive LLMs (which often have hundreds of billions). By design, SLMs can run on consumer-grade devices like smartphones, embedded systems, or PCs, offering fast inference and a much lower cost. Researchers often consider models under about 10 billion parameters to be SLMs, since such models can fit on common hardware with low latency.

In June of 2025, Nvidia research released a paper detailing the potential of SLMs, titled “Small Language Models are the Future of Agentic AI.” One of the key takeaways from the paper is that since agents are typically tailored towards solving very specific tasks, a full hundred billion parameter LLM is not required to be proficient at the task. They show that SLMs are in fact enough for specific agentic applications with examples in specific industries. SLMs use many state-of-the-art optimization techniques and fine-tuning to decrease the model size and improve efficiency. Some of these techniques allow SLMs to be decently powerful and useful at small sizes. Techniques include quantization, mixture-of-experts (MoE), low rank adaptation (LoRA), pruning, flashattention and more.

🔅 AI Product Foundations: Planning Strategies for Data Scientists 📝 Discover planning strategies, frameworks, and practical skills to build AI products as a data scientist. 🌐 Author: Matthew Blasa 🔰 Level: Intermediate ⏰ Duration: 1h 44m 📋 Topics: Data Science, Artificial Intelligence, Product Strategy 🔗 Join Artificial intelligence for more courses