Press release:

Arabic.AI, the regional leader in Arabic artificial intelligence and enterprise technology solutions, today announced a collaboration with Stanford University’s Center for Research on Foundation Models (CRFM) to establish the first holistic benchmark for evaluating Arabic large language models (LLMs). The project represents a significant milestone in global AI research and ensures that Arabic receives the same level of rigorous evaluation as other major world languages.

Stanford’s CRFM is recognized for pioneering the HELM (Holistic Evaluation of Language Models) framework, an open-source platform that provides transparent and reproducible benchmarks for assessing the capabilities and risks of foundation models. By extending HELM into Arabic, this project will provide the Arabic AI community with a trusted reference point to measure the strengths and limitations of different models.

For Arabic.AI, whose Arabic.AI LLM-X (Flagship) and Arabic.AI LLM-S (Small) model are among the most advanced Arabic-first LLMs, the collaboration reflects its mission to drive innovation in Arabic AI while contributing to a public good that benefits the wider research and enterprise ecosystem.

“Arabic is spoken by more than 400 million people, yet it has historically been underserved in AI benchmarking,” said Nour Al Hassan, CEO of Arabic.AI. “This collaboration with Stanford’s CRFM ensures that Arabic is evaluated with the same rigor, transparency, and visibility as other global languages. It is a step forward not just for Arabic.AI, but for the entire Arabic AI community.”

The first phase of the project, including the Arabic leaderboard built on the HELM framework and the new evaluation methods for conversational AI, has now been completed. This gives researchers and enterprises a clear and reliable foundation for understanding model performance in Arabic. This work sets the stage for broader efforts that will advance Arabic AI on the global stage.

Read more about what HELM Arabic evaluates and how it works on the HELM Arabic Page at Stanford's CRFM Website.

About Arabic.AI

Arabic.AI is a leading provider of Arabic artificial intelligence and enterprise language solutions. Arabic.AI develops Arabic-first AI technologies, including its flagship Arabic.AI LLMs, designed to transform translation, content, and enterprise operations at scale.