The Allen Institute for AI has released Olmo-core 3, an open-source framework for training large mixture-of-experts language models, and says it trains such models about 2.7 times faster than Nvidia's widely used Megatron-core software. SiliconANGLE reported the release Friday. The code is on Ai2's GitHub, accompanied by a white paper describing the system.

What Happened

Mixture-of-experts, or MoE, models split their parameters into many specialized sub-networks, called experts, and route each token through only a few of them. That keeps the cost of each prediction down even as total model size grows, but it makes training harder to organize across many GPUs.

Olmo-core 3 supports expert pools that scale from eight to 128 experts, with four experts active for each token. Ai2 says the same infrastructure can carry models past 1 trillion parameters.

RelatedNew curl Release Fixes 24 Flaws, Including a High-Severity Bug Reported With Anthropic’s Claude

The framework relies on several techniques to spread the work. Expert parallelism places different experts on different GPUs so no single card has to hold all of them. Model layers can also be distributed across groups of GPUs, cutting how much each device stores. A distributed optimizer splits optimizer state across GPUs instead of copying it onto every card. Olmo-core 3 also supports MXFP8, a reduced-precision number format that lowers both computation and the volume of data moved between GPUs.

In Ai2's benchmark, the framework trained a 47-billion-parameter model at about 52,000 tokens per second on Nvidia GPUs. Megatron-core reached roughly 19,400 tokens per second on the same task, according to the figures SiliconANGLE reported, which works out to the 2.7-fold difference.

Why It Matters

Training infrastructure is one of the main barriers for open-model developers outside the largest AI labs. Frontier-scale MoE models are typically trained with in-house software at companies that have both large GPU clusters and engineering teams to tune them. Ai2 says its goal is to put trillion-parameter training within reach of researchers who lack government or big-enterprise infrastructure.

Throughput gains translate directly into cost. If a team can process nearly three times as many tokens per second on the same hardware, it needs fewer GPU-hours to reach a given amount of training data, which matters most for groups paying for rented compute.

Ai2 also says the framework lets researchers adapt MoE training to different hardware and experiment with routing and parallelism strategies, areas where design choices are often locked inside proprietary training stacks.

Context

Ai2, the Seattle research nonprofit founded by the late Microsoft co-founder Paul Allen, publishes its Olmo language models with training data, code and intermediate checkpoints, positioning them as fully open rather than open-weight only. Olmo-core is the training library underneath that work.

Megatron-core, Nvidia's open-source library for large-scale transformer training, is a common baseline for such comparisons. A single benchmark on one model size, run by the framework's developer, does not establish how the two compare across configurations, and performance can shift with GPU generation, interconnect and model architecture.

What to Watch

The main test is outside replication: whether other open-model teams adopt Olmo-core 3 and report similar speedups, and how it performs at the trillion-parameter scale Ai2 is targeting rather than at 47 billion. A future Olmo model trained on the framework would be the clearest signal of what it can do in practice.