The development of large language models (LLMs) for low-resource languages has progressed from a largely theoretical challenge to an active area of empirical research. This talk uses Basque as a case study to examine two open problems that extend to low-resource languages more broadly: how to build instruction-following LLMs for languages with limited resources, and how to evaluate them. Building on Latxa (a family of open Basque LLMs), we present two complementary approaches to instruction-tuning, fine-tuning and model merging, which we validate through a combination of standard benchmarks, verifiable instruction following, and a community evaluation arena. The second part of the talk frames evaluation itself as an open research problem, presenting ongoing work on cross-lingual benchmarking and human evaluation methodology, and discussing what the latter reveals about the challenges of assessing linguistic quality in low-resource settings.
Naiara Perez is a postdoctoral researcher at the HiTZ Basque Center for Language Technology (IXA Group) at the University of the Basque Country, where she works on natural language processing for Basque and related low-resource languages. Her research focuses on building LLMs for low-resource languages and on the methodological challenges of evaluating them. She was a central contributor to Latxa, the largest family of open Basque language models, where she led the curation of pretraining corpora and the development of language-specific benchmarks. Her current work examines instruction-tuning for low-resource languages and human evaluation methodologies. She is also a core participat in ALIA and ILENIA, the primary national initiatives funded by the Spanish government to develop foundation models and linguistic resources for Spain's official languages.