STORY CREDITS
Writer: Vanajakshi B.H.
Photo: Arijit Reeves
The Lingo Research Group at IIT Gandhinagar proudly presents Ganga-1B, a breakthrough in language models. Named after the longest river flowing through the Hindi-speaking region of India, Ganga-1B is the first pre-trained Hindi model developed by an academic research lab in India.
Project Unity aims to celebrate and harness India’s rich linguistic diversity by creating a comprehensive resource for the country’s major languages. The initiative strives to achieve state-of-the-art performance in understanding and generating text in Indian languages. Our first milestone is the release of the Ganga-1B model, trained on an extensive monolingual Hindi language dataset.
The Ganga-1B model has been meticulously trained on a large dataset of public domain web-crawled Hindi language data. This includes news articles, web documents, books, government publications, educational materials, and quality-filtered social media conversations. Native Indian speakers have further curated the dataset to ensure high quality. Impressively, Ganga-1B outperforms existing open-source models supporting Indian languages, even those with up to 7 billion parameters.
Key Features:
- Developed by: Lingo Research Group at IIT Gandhinagar
- Model Type: Autoregressive Language Model
- Languages: Bilingual (Primary: Hindi [hi], Secondary: English [en])
- License: Apache 2.0
Technical Specifications:
- Precision: Float32
- Context Length: 2,048
- Learning Rate: 4e-4
- Optimizer: AdamW
- LR Scheduler: Cosine
Model Architecture and Objective: Ganga-1B is a decoder-only transformer model with the following specifications:
- Layers: 16
- Attention Heads: 32
- Embedding Dimension: 2,048
- Vocabulary Size: 30,000
- Sliding Window: 512
- Intermediate Dimension: 7,168
The team took nearly 1.5 years to develop the Ganga-1B model using open-source data from various websites. Ganga-1B is open source and has already been downloaded by over 600 people in less than 48 hours after the announcement. Furthermore, the research team is working on models for other languages, including Tamil, Telugu, Marathi, Gujarati and Urdu. They are also exploring the use of AI in e-governance for regional languages. To support school students and teachers, the team is working on an education LLM . If someone is looking for develop chatbot in Hindi, why wait? Take advantage of this free model today.
For More details can contact: lingo@iitgn.ac.in
About IITGN: IIT Gandhinagar (IITGN): founded in 2008 and located in Palaj, Gandhinagar, Gujarat, offers a distinctive undergraduate and graduate education with innovative curricula that emphasize critical thinking, interdisciplinary knowledge, and a liberal arts approach. The institute’s student-centric philosophy fosters a safe, nurturing, and empowering environment, promoting project-oriented learning, compulsory design and life sciences courses, and global diversity. IITGN’s five-week Foundation Programme for new undergraduates earned the World Education Award 2013 for innovations in engineering education. Over 40% of students gain international study experience, reflecting IITGN’s commitment to excellence in science, technology, humanities, and social sciences. As India’s first 5-star GRIHA LD (Green) campus and the first 5-star campus for food safety and healthy eating, IITGN also upholds exemplary safety standards for its construction workforce, with practices recommended by the IIT Council for adoption across all IITs. To know more please visit https://www.iitgn.ac.in