Vambo AI built MORENA for 12 African languages without real data

Johannesburg-based Vambo AI trained MORENA, a 1.5-billion-parameter model covering 12 African languages, but struggled to find real text and relied heavily on synthetic data. Training on 251.7 billion tokens ran on the Leonardo supercomputer via a UNDP-backed programme at no cash cost.

• MORENA covers 12 languages including ChiShona, Kiswahili, Hausa, Yorùbá, Igbo and isiZulu
• Training used 22,000 A100 GPU-hours worth an estimated $25,000–$40,000
• MORENA scored 1.408 bits per byte, the lowest among 26 models tested
• Weights are released under Apache 2.0, with 0.5B and 0.2B versions for low-resource use