Why is it difficult for AI language tools to learn Latvian? ChatGPT explains: ”There is not enough data and text in Latvian. The Latvian language has a complex grammar and language structure – nine conjugations, different genders, and figures – with multiple dialects and regional differences. It is one of the less common languages in the world making it not a priority for AI models.”

The vast majority of Large Language Models (LLM) such as ChatGPT are predominantly trained on English data sets, simply because this is the lingua franca of the Internet. Different major – mostly European – languages such as German, French, and Spanish are also well represented. Other languages, on the other hand, are not, leading to <a href=”https://www.economist.com/science-and-technology/2024/01/24/why-ai-needs-to-learn-new-languages” style=”text-decoration: underline !important;”>less representation of the worldviews and cultures that these languages carry with them.

Now, Tilde, a Latvian language technology company, has just won the European Commission's Large AI Grand Challenge, which will allow them to develop a foundational LLM for European languages. It will focus particularly on underrepresented European languages, especially Eastern European and Baltic languages. These languages are ”poorly covered in the current models”, the company stated following their win. They also claim that their model will improve AI applications for more than 155 million Europeans.