New Model Release
At the recent India-AI Impact Summit 2026, Sarvam AI, a prominent Indian AI startup, proudly announced the availability of its two sophisticated, foundational
AI models. These models, boasting 30 billion and 105 billion parameters respectively, are now accessible to the public under an open-source license. Developed entirely in-house, these large language models (LLMs) were trained on extensive, meticulously curated datasets. The computational power for this ambitious training initiative was facilitated by GPUs acquired through the Indian government's substantial Rs 10,372-crore IndiaAI Mission, with essential infrastructure support provided by data center operator Yotta and technical expertise from Nvidia. Sarvam has made these models available for commercial applications via the Apache 2.0 license, with their weights downloadable from both AIKosh and Hugging Face platforms. Additionally, users can interact with these models through Sarvam’s Indus AI chatbot app and its developer portal.
Architectural Innovations
Delving into the technical core of Sarvam's new models reveals an advanced Mixture-of-Experts (MoE) transformer architecture. This design is particularly efficient, as it only activates a subset of the model's vast parameters for any given task, thereby significantly reducing computational demands and costs. The smaller, 30-billion-parameter model is equipped with a 32,000-token context window, making it adept for real-time conversational applications. The more substantial 105-billion-parameter model offers an expanded 128,000-token window, which is ideal for handling intricate, multi-step reasoning processes. For enhanced efficiency, the Sarvam 30B model incorporates Grouped Query Attention (GQA) to minimize KV-cache memory usage without compromising performance. The Sarvam 105B model, conversely, utilizes a DeepSeek-inspired Multi-head Latent Attention (MLA) mechanism, further optimizing memory requirements for extended context processing.
Training Data & Performance
The training datasets for both Sarvam models encompass a diverse range of content, including programming code, general web data, specialized knowledge bases, mathematical information, and extensive multilingual content. A significant portion of the training resources was dedicated to cultivating a rich multilingual corpus, specifically focusing on the ten most widely spoken languages in India. Preliminary benchmark results indicate that the 105B model outperformed its 30B counterpart during the early training phases, suggesting effective scaling capabilities. When compared to other LLMs of comparable size, the 105B model achieved performance levels on par with models like pt-oss 120B and Qwen3-Next (80B) in general tasks. It also demonstrated superior capabilities in agentic reasoning and task completion, surpassing DeepSeek R1, Gemini 2.5 Flash, and o4-mini on the Tau 2 Bench. However, for code generation tasks, the 105B model's performance on SWE-Bench Verified lagged behind comparable models. The 30B model showed a slight edge over Nemotron 3 Nano 30B in coding (SWE-Bench Verified) and agentic reasoning (Tau2), though it performed slightly less well on benchmarks like Live Code Bench v6 and BrowseComp.
Indic Language Prowess
A key differentiator for Sarvam's models is their exceptional performance in Indian languages, largely attributable to a custom-built tokenizer. This tokenizer was developed and trained from scratch to ensure highly efficient processing of text across all 22 scheduled Indian languages, which utilize 12 distinct scripts. The tokenizer's effectiveness is quantified by its fertility score, representing the average number of tokens needed to encode a word. In this regard, Sarvam's tokenizer significantly outperformed other open-source alternatives in efficiently encoding Indic text. This tailored approach to tokenization is crucial for capturing the nuances of Indian languages and ensuring these models are truly effective for local applications, aligning with India's broader 'sovereign AI' ambitions.
Safety and Alignment
Sarvam has placed a strong emphasis on safety and ethical considerations throughout the model development process. During the supervised fine-tuning stage, both the 30B and 105B models were trained on a dataset specifically designed to address both standard and India-centric risk scenarios. This comprehensive dataset included adversarial prompts and 'jailbreak' attempts, meticulously collected through automated red-teaming exercises. Each of these challenging prompts was paired with policy-aligned and safe completions, ensuring that the models learn to respond in a secure and responsible manner, reinforcing their reliability for diverse applications.
