← Back to Feed

Post from 17 October 2026

In my last post, I wrote about Beam, the open model from Reflection AI. One number in its announcement is easy to misread: it uses 23B of its 501B parameters for each token it processes.

For those not familiar with this, parameters are the numbers a model learns in training. These billions or even trillions numbers determine how the model processes inputs and generates outputs (tokens). A token is a word or piece of a word. To run the model, they have to be loaded into GPU memory.

Reflection AI claims that Beam needs less computing power to run comparing to other models due to the nature of how it’s built.

In a traditional “dense” AI model, every single part of the neural network activates for every prompt, regardless of whether you are asking it to write a poem or solve a complex math equation.

Beam uses a “mixture-of-experts” (MoE) architecture where it’s divided into several smaller, specialized sub-networks, called experts. When you give it a prompt, a “router” quickly analyzes your request and sends it only to a few experts best suited to handle that specific topic. The rest stay inactive. By only lighting up a fraction of its total “brain” for any given task, an MoE model can have the vast knowledge capacity of a massive model but operate with the speed and computational lightness of a much smaller one.

So the math per token is roughly a 23B model’s. But the router chooses again for the next token, and it can pick any expert, so all 501B still have to stay loaded.

The 23B is your compute bill. The 501B is your memory bill. Reflection AI’s claim, which is based partly on estimates rather than measured serving cost, is about the first one - the compute cost.

With that being said, you can shrink the memory cost by storing each number with fewer bits (quantizing), or keep idle experts in slower memory and pull them in when needed (offloading). The first costs some precision and the second costs speed.

So if you’re estimating hardware from a spec sheet that leads with “active parameters,” don’t mistake that number for the model’s size. It’s the “computing per token.” Make sure to size your GPU memory from the total.

Also on Bluesky · Mastodon

Loading comments…