Ƶ

Training an Arabic LLM that reflects local values

Training an Arabic LLM that reflects local values

Training an Arabic LLM that reflects local values
The Arab world did not play a key role in the PC, internet and mobile eras. In the AI era, it will be different. (Shutterstock)
Short Url

Advances in the large language models that underpin generative AI are changing everything, from medicine and education to entertainment.

Our relationship with technology is becoming more intimate as machines change from passive tools into active assistants that amplify our innate human abilities.

This new era poses both a challenge and an opportunity for the Middle East.

The challenge is that leaders in this new field, like OpenAI’s ChatGPT and Google’s Gemini, come from Silicon Valley, or from China, where my team at 01.AI has built models that rival the Americans. In Europe, too, startups such as France’s Mistral have entered the race.

The opportunity is for the Middle East to join this league and make sure its voice is heard.

Inspired by my latest trip to Riyadh, I decided to test how the current crop of AI models would handle a simple request. I imagined myself as a young Saudi getting ready to host a dinner party and asked ChatGPT to prepare a menu.

The food it recommended sounded delicious — stuffed grape leaves, tabouleh salad, mandi and stuffed dates. But the beverages were a problem.

Aside from drinks such as mint lemonade and jallab, a mixture of dates, grape molasses and rose water, ChatGPT also offered this: “For alcoholic beverages, you could offer a selection of international wines, beers, or non-alcoholic mocktails.”

To its credit, when I repeated the question, it offered only non-alcoholic drinks.

If a model recommends breaking both the law and cultural norms, imagine how it might answer other more sensitive questions about politics or religion? Indeed, researchers have even shown that some models have exhibited an anti-Muslim bias.

My modest test underlines the urgent need to develop an Arabic large language model that reflects local values.

The first step to building this is creating enough high-quality Arabic digitized data to properly train a new generation of models.

Although there are 400 million Arabic speakers, only an estimated 2 percent of online content is in Arabic. Meta’s open source LLM model Llama is overwhelmingly trained on English data, with Arabic comprising less than 0.1 percent of the data.

The lack of data naturally skews the results. To fix this dearth of data, either a visionary entrepreneur or a government-backed organization should collect, digitize and convert the many Arabic books into training data for Arabic models.

Once the data is gathered, it can be fed into the breakthrough pre-training process, which reads trillions of words and creates its own virtual concept space or model of the world. This concept space has been shown to be mostly in English and Chinese.

Adding a sizable number of texts in Arabic, which has enormous cultural output and significance, will make the concept space more knowledgeable about Arabic and more balanced in its concepts and views.

After such pre-training, the model needs to be fine-tuned by data and labels from the Arab world, which will align with the values of the region. Those are different from American models, which are aligned to US values, and Chinese models, which reflect Chinese values.

The collection of alignment data, the coordination of human labeling and the alignment process will need to be done in-region by AI experts.

A new Arabic-enhanced large language model could encourage entrepreneurs and developers to build new applications tailored to the needs of their nations.

Kai-fu Lee

Finally, safety modules will need to be added to ensure legal compliance and to avoid harm. These will also need to be developed locally.

The above steps will create localized, sovereign models that will reflect the traditions of the Middle East. Privately developed or government-backed, it could be the foundation for a new wave of Arabic AI innovation.

A new Arabic-enhanced large language model could encourage entrepreneurs and developers to build new applications tailored to the needs of their nations.

Imagine an AI tool that could find, summarize, organize and write insightful content, an AI teacher that makes learning fun and customized, an AI doctor that is more knowledgeable than any human, an AI engineer that can write software and applications, and an AI assistant that knows its owner better than the owner themselves.

The Arab world did not play a leading role in the PC, internet and mobile eras. In the AI era, it will be different.

This transformation is by no means an easy feat. It will require an unprecedented investment of money, energy and human capital.

Middle Eastern leaders like Saudi Crown Prince Mohammed bin Salman and others have shown that they have the vision, determination and resources to lead their countries into the future.

Standing on my hotel balcony in Jeddah recently, overlooking the King Abdullah University of Science and Technology, I saw part of that vision coming to fruition.

Universities such as KAUST and the Mohamed bin Zayed University of Artificial Intelligence in the UAE are striking examples of the resources that have already been poured into this transformation.

These world-class academic institutions can attract and retain the best top tier global talent.  It is especially important to bring in the world’s best computer engineers to help fulfill this vision of the future AI.

Our team at 01.AI has shown what a group of talented and motivated computer scientists can achieve in just one year. With the right commitment of resources and drawing upon the best talent, countries like Ƶ can easily catch up with their global peers.

The Middle East can also lead the world in the use of renewables to run power-hungry generative AI models.

As it seeks to diversify its economy, Ƶ is actively promoting the use of alternative energy sources such as solar, which could power server farms and reduce their carbon footprint — a growing concern as AI becomes more widespread.

It may take time for countries to figure out their strategy for building a sovereign AI. But it is critical for the Arab world to quickly catalyze the creation of culturally appropriate LLMs and build a rich ecosystem to allow AI-powered Arabic apps to blossom.

A recent encounter with a female sales assistant at a computer store in Riyadh served as an apt reminder of what is at stake. Dressed in jeans and sporting a tattoo, she was a reminder of the transformative changes that the country is undergoing.

Where are you from, I asked. “I’m Saudi,” she said. “One day I want to be Ƶ’s Elon Musk.” I hope on my next visit she will pitch me a homegrown AI app.

Kai-Fu Lee is a computer scientist, CEO of 01.AI, chairman of Sinovation Ventures, former president of Google China, and author of “AI 2041” and “AI Superpowers”
 

Disclaimer: Views expressed by writers in this section are their own and do not necessarily reflect Arab News' point of view

Ƶ’s official reserves reach $457bn, up 4%

Ƶ’s official reserves reach $457bn, up 4%
Updated 1 min 57 sec ago

Ƶ’s official reserves reach $457bn, up 4%

Ƶ’s official reserves reach $457bn, up 4%

RIYADH: Ƶ’s official reserve assets reached SR1.71 trillion ($456.97 billion) in September, marking a 4 percent increase year-on-year, according to new data.

Figures released by the Saudi Central Bank, known as SAMA, show these holdings include monetary gold, special drawing rights, the International Monetary Fund’s reserve position, and foreign reserves.

The latter, comprising currency and deposits abroad as well as investments in foreign securities, made up 94.5 percent of the total, amounting to SR1.62 trillion in September. This category grew 4.11 percent during this period.

September data indicated that special drawing rights rose to SR79.86 billion, marking a 4.18 percent increase and reaching the highest level in two and a half years. SDRs now account for 4.66 percent of Ƶ’s total reserves.

Created by the IMF to supplement member countries’ official reserves, SDRs derive their value from a basket of major currencies, including the US dollar, euro, Chinese yuan, Japanese yen, and British pound sterling. They can be exchanged among governments for freely usable currencies when needed.

SDRs provide additional liquidity, stabilize exchange rates, act as a unit of account, and facilitate international trade and financial stability.

The IMF reserve position totaled around SR12.64 billion, but decreased by 11.45 percent during this period. This category represents the amount a country can draw from the IMF without conditions.

Ƶ’s official reserves have been a fundamental pillar of the nation’s economic stability and are closely tied to its strategic investments in foreign securities.

The Kingdom’s reserves include an extensive portfolio of foreign assets, diversified across currencies and geographies, ensuring the country has a robust financial buffer against global economic uncertainties.

This prudent reserve management has helped Ƶ maintain a resilient fiscal position and a strong credit rating, affirmed at “A/A-1” by S&P Global, which recently upgraded the Kingdom’s outlook to positive due to its sustained reform momentum.

In alignment with Vision 2030, Ƶ has adopted an expansionary fiscal policy to support transformative projects aimed at reducing its economic dependence on oil.

This ambitious agenda has led to budget deficits and prompted the country to tap into debt markets to finance key infrastructure and social initiatives.

Despite the uptick in debt, the Kingdom remains fiscally well-positioned, with ample reserves and substantial net assets, projected to stay above 40 percent of GDP through 2027 according to S&P Global.

This buffer underscores Ƶ’s capacity to absorb potential economic shocks while continuing to pursue its development goals.

The nation’s significant reserve base not only underpins its economic stability but also provides the flexibility to recalibrate spending on large infrastructure projects as needed, maintaining a balance between growth and fiscal discipline.

This strategy is essential as Ƶ seeks to nurture its non-oil sectors, supported by the Public Investment Fund and other governmental entities.

The PIF’s role in fostering a diversified economy is central to Vision 2030’s objectives, from investment in renewable energy to technology and healthcare, creating a more resilient and diversified economic base.

With the positive outlook and strategic focus on sustainable growth, Ƶ’s economic reforms are expected to drive strong non-oil growth over the medium term, further cementing the Kingdom’s fiscal stability and enhancing investor confidence in its long-term economic vision.


COP29: Clean energy a catalyst for stability, recovery in conflict zones

COP29: Clean energy a catalyst for stability, recovery in conflict zones
Updated 12 min 30 sec ago

COP29: Clean energy a catalyst for stability, recovery in conflict zones

COP29: Clean energy a catalyst for stability, recovery in conflict zones
  • Environmental solutions reduce dependence on imports
  • Micro-grids support conflict-ridden communities

BAKU: As COP29 progresses in Baku, attention is turning to the ways in which clean energy can transform post-conflict recovery efforts, bringing both environmental resilience and social stability to regions affected by war.

This year’s discussions have highlighted how renewable energy offers more than environmental benefits, having the potential to catalyze economic recovery, improve living standards and build long-term resilience in areas most vulnerable to conflict.

Renewable energy in conflict recovery: A new dimension of aid

Experts have highlighted how sustainable infrastructure can reduce dependence on foreign energy imports and fuel local economies in war-torn areas.

Hafed Al-Ghwell, a North African geopolitics expert, said in an interview with Arab News that “clean energy isn’t just about generating power; it’s about autonomy and resilience.” For regions dependent on volatile foreign fuel supplies, renewables offer a more stable power source that strengthens local autonomy.

Gilles Carbonnier, vice president of the International Committee of the Red Cross, highlighted the critical role of renewable energy in supporting communities severely affected by both conflict and climate change.

“The people who are most affected by climate change risks are those who live in zones of armed conflict and have the least capability to adapt and face these risks,” Carbonnier said.

He described how the ICRC is using solar power to help protect communities from droughts, floods and extreme weather across the Sahel, the Horn of Africa and the Middle East.

“What we need is to scale these efforts, which means directing much more climate funding to conflict zones,” Carbonnier added.

This local approach provides immediate aid while laying the foundation for sustainable recovery in areas struggling with limited resources and infrastructure damage.

Gaza: The intersection of war and environmental crisis

The war and occupation in Gaza represents a severe environmental and humanitarian crisis.

Crown Prince Hussein of Jordan addressed COP29. In calling for global solidarity with Gaza, he said: “Saving our planet must start from the premise that all lives are worth saving.” He described how the war is “compounding environmental challenges for Gaza and beyond.”

A recent UN Environment Program report highlighted severe contamination of Gaza’s land, water and air due to the destruction of critical infrastructure, including sewage and waste systems, leaving communities surrounded by hazardous debris.

Carbonnier said that Gaza is emblematic of the dual crisis faced by many conflict zones, where war intensifies environmental damage and deepens humanitarian challenges.

“In Gaza, conflict has degraded critical infrastructure to the point where basic resources like clean water and electricity are scarce,” he said.

“Renewable energy solutions, such as solar micro-grids, could offer essential relief by providing stable power to hospitals, schools and homes,” he added.

In Gaza, solar micro-grids deployed by NGOs are already providing essential power for hospitals and emergency shelters, offering a sustainable alternative to fuel imports which have been blockaded by Israeli forces since the conflict began.

An image from the COP29 conference in Baku. AN

Resilience through clean energy infrastructure

Renewable energy infrastructure, particularly solar and wind power, is highly adaptable to conflict and post-conflict settings due to its low maintenance requirements and modular design.

Solar panels and wind turbines require minimal upkeep and their modular nature allows for incremental infrastructure development as security improves.

This approach has proved effective in Syria, where solar-powered micro-grids are supplying power to refugee camps, providing consistent electricity for vital services like sanitation and healthcare.

According to Carbonnier, these micro-grids “reduce dependence on often costly and dangerous fuel deliveries and stabilize power supplies for communities under stress.”

Renewable energy micro-grids are now recognized as a cornerstone of humanitarian aid, offering stability to populations affected by protracted crises.

Policy implications and international support

For renewable energy to become a reliable tool in post-conflict recovery, coordinated international support and robust policy frameworks are essential.

Azerbaijan’s lead COP29 negotiator, Yalchin Rafiyev, highlighted the need for financial support specifically directed at conflict zones. “Bridging the gaps between climate finance and peace-building efforts can unlock substantial benefits for communities emerging from conflict,” Rafiyev said.

Rumen Radev, president of Bulgaria, highlighted the link between climate resilience and global stability, telling Arab News: “Extreme meteorological events threaten not just people and economies, but also the security and stability of the world.”

His remarks highlight the importance of COP29’s goals in fostering peace through enhanced climate resilience.


Daesh group gunmen kill politician in Pakistan

Daesh group gunmen kill politician in Pakistan
Updated 25 min 28 sec ago

Daesh group gunmen kill politician in Pakistan

Daesh group gunmen kill politician in Pakistan
  • Attackers escaped after shooting the Islamist politician in Bajaur district, near the border with Afghanistan where militants remain active

PESHAWAR, Pakistan: Gunmen from the regional branch of the Daesh group have killed a politician in northwest Pakistan, police and the militants said Friday.
“Jamaat-e-Islami Bajaur leader Sufi Hameed was leaving the mosque after offering prayers after sunset (Thursday) when two masked men on a motorcycle opened fire on him,” senior police official Waqar Rafiq said.
The official said the attackers escaped after shooting the Islamist politician in Bajaur district, near the border with Afghanistan where militants remain active.
Islamic State Khorasan (IS-K) said its “soldiers shot an official of the apostate political party,” in a message on Telegram.
The local chapter of the Daesh group accuses religious political parties of going against strict religious preachings and supporting the country’s government and the military.
IS-K has recently carried out several attacks against political parties, including a suicide bomb blast at a rally in Bajaur last year which killed at least 54 people including 23 children.
“In this year alone, they have killed at least 39 people in targeted attacks and bomb explosions” in Bajaur, a senior local security official said on the condition of anonymity.
In both Khyber Pakhtunkhwa province, where Bajuar is located, and Balochistan province in the southwest, armed Islamist or separatist groups regularly target security forces and state representatives.
Militants operating in Pakistan include Tehreek-e-Taliban Pakistan (TTP), the country’s homegrown Taliban group.
Pakistan has seen a sharp rise in militant attacks in regions bordering Afghanistan since the Taliban returned to power in the country in 2021.


Fire breaks out at a Spanish nursing home, killing at least 10 people

Fire breaks out at a Spanish nursing home, killing at least 10 people
Updated 35 min 52 sec ago

Fire breaks out at a Spanish nursing home, killing at least 10 people

Fire breaks out at a Spanish nursing home, killing at least 10 people
  • Authorities were alerted of the blaze early Friday morning in Villa Franca de Ebro
  • Fire took place just weeks after devastating flash floods in Valencia killed more than 200 people

MADRID: At least 10 people died in a blaze at a nursing home in Zaragoza, Spain, before firefighters managed to extinguish it, local authorities reported on Friday.
Authorities were alerted of the blaze early Friday morning in Villa Franca de Ebro, about 30 minutes from the northeastern city.
The cause of the fire was not yet known, local media reported.
Jorge Azcon, head of the regional government of Aragon, whose capital city is Zaragoza, confirmed the deaths and said on X, formerly Twitter, that all government events in the region were canceled for the day.
Spanish Prime Minister Pedro Sanchez also expressed his shock over the fire and deaths.
The fire took place just weeks after devastating flash floods in Valencia killed more than 200 people and destroyed thousands of homes. The floods were the worst natural disaster in Spain’s recent history.


Ƶ fight back to stun Germany in NEOM Beach Soccer Cup 2024

Ƶ fight back to stun Germany in NEOM Beach Soccer Cup 2024
Updated 53 min 13 sec ago

Ƶ fight back to stun Germany in NEOM Beach Soccer Cup 2024

Ƶ fight back to stun Germany in NEOM Beach Soccer Cup 2024

NEOM: Hosts Ƶ threw off their opening day defeat against England with a 7-6 win over Germany in the second day of action at the NEOM Beach Soccer Cup 2024.

Spurred on by the support of their fans at Gayal Beach, the national beach soccer team took to the sand on Thursday night.

Al-Hamami and Qasem gave Ƶ a 2-0 lead in the first period and the hosts added two more goals to their tally in the second, Salman and Waleed both finding the net. Germany then closed the gap as Peterson scored a brace.

There was even more action in the third period, as four goals from Germany and two from the home side took the match into extra time at 6-6.

It was left to Waleed to score the winner and reward Ƶ with two points on the board following their first win in this year’s competition.

Also in Group A, after securing a first-day victory against Germany, the UAE struck three times in the first 12 minutes against England with goals from Ali, Kamal and Mousab.

England responded well, finding the back of the net three times in the space of two minutes during the second period courtesy of a Robinson brace and one from Lawson. Despite the setback, A. Abbas restored the advantage for his team before the period ended.

Both teams were on target twice in the third and final period, A. Abbas scoring his second and goalkeeper Bahri hitting the target for the UAE. Lawson got his second for England, with Younie also striking home. But it was not enough, and the UAE clinched their second victory of the tournament to put them on maximum points.

The second Group B match of the day saw a repeat of the NEOM Beach Soccer Cup 2023 final as defending champions Brazil took on Japan.

The scores were even at 2-2 by the end of the first period, as Antonio scored two for Brazil, and Oba and Akaguma found the net for Japan. The teams were tied in the second phase of play too, Benjamin Jr. and Oba scoring within six seconds of each other.

Brazil secured victory in the third and final period with a brace from Edson Hulk and a goal from Datinha, although they conceded goals from Akaguma and Oba.

Eight teams are taking place in the men’s tournament during this third edition of the cup. Group A is made up of Ƶ, the UAE, England and Germany, with Brazil, Spain, Japan and China in Group B.

In the women’s competition, defending champions Spain, 2022 participants England and Ukraine, and 2022 champions Brazil will take to the sand.