GLOSSARY: A Must for Indic Language Models

пятница, 9 февраля 2024 г.

A Must for Indic Language Models

Can't read or see images? View this email in a browser

By AMIT RAJA NAIK

Friday, Feb 9, 2024 | Was this email forwarded to you? Sign up here

___________________________________________________________

A few months ago, AIM pointed out the dire need for creating benchmarks for Indian languages since most of the famous ones, like MMLU and HumanEval, do not necessarily include a good amount of datasets for Indian languages. Now with Indic LLMs coming into the picture, it is time that India got its own LLM benchmark and leaderboard for its models.

At MLDS 2024, Tamil Llama creator Abhinand Balachandran highlighted the importance of creating benchmarks specifically tailored for evaluating Indic language models. Similarly, in an exclusive interview with AIM, Shantipriya Parida, the creator of Odia Llama, revealed that he was planning to create a benchmark for Indic language models.

While Indic models do not currently have a benchmark such as the Hugging Face Open LLM Leaderboard, there are several evaluation datasets available where creators can test their model on the provided dataset.

Some of them include AI4Bharat’s IndicSentiment dataset, which was used to evaluate Airavata, the recent Indic language model and Vistaar, a benchmark and training set for Indian languages for ASR, among others. Interestingly, there is also a lesser-known benchmark on Hugging Face, called IndicBenchmarkData, created by Sambit Sekhar, which includes the Indic benchmark dataset for Gujarati, Bengali, Telugu, Tamil, and several other Indic languages.

What about the leaderboard? Read the full story here.

Developing Proprietary Generative AI Models

AIM recently caught up with Partha DeSarkar, Group CEO at Hinduja Global Solutions (HGS), the IT management division of the billion-dollar Hinduja Group, who spoke at length about building proprietary generative AI models, alongside its adoption, and impact in the Indian IT landscape.

Read the full story here.

Healthcare x LLMs

Healthcare was already going through tough times in AI adoption, and now LLMs just got added to that list. Many experts have shared mixed emotions when it comes to LLM adoption because of the notable downsides like providing false information, manipulating data, and promoting plagiarism.

See how LLM is breathing in the healthcare space here.

Is Meta the new OpenAI?

Both OpenAI and Meta share one common goal: to achieve AGI quickly. One benefits immensely from open source but rarely acknowledges it, while the latter focuses more on contributing and exploring its potential, hoping that no adverse outcomes occur. Read more here.