An AI agent is a language model in a loop, so it’s easy to assume every step in it needs one. But look at what those steps ask: many aren’t asking the model to write anything, just to pick.
Post from 26 October 2026
Take a simple agent loop. It works out what the user wants. It chooses the next action: search the database, call an API, ask the user, or stop. Then it checks the result and either continues or retries.
The second and third steps end in a pick from a short list you could write down in advance. So do a lot of the other questions agents send to an LLM: is this document relevant, did that tool call succeed, which specialist agent should take this task. Those are classification and routing problems (sorting an input into one of a few known answers).
You can still use an LLM for them, and it may work well. But then you’re using a model built to write as a classifier, and it can cost more and take longer than a pick needs, especially if it reasons through the question before it answers.
That’s the argument behind Jev, which TypeSafe AI released in early access in September: a model that only decides. You give it the state and the options, and it returns probabilities over those options instead of text.
The LLM may well stay on some of those steps. The difference is that you picked it for a decision on purpose. One boundary I’d keep in mind: a short answer isn’t the same as an easy question. “Is this document relevant?” is yes or no, but getting it right can take real understanding. Whatever you hand a step to, test it on that step.
So go through your agent workflow and mark every step whose answer comes from a list you could write down.
Loading comments…
Comments aren’t loading right now.
The post is unaffected; the comment service didn’t respond. Try again in a moment, or come back later.