<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>AI for the Curious · Feed</title>
    <link>https://aiforthecurious.ai/feed</link>
    <description>What’s caught my attention in AI lately.</description>
    <language>en-us</language>
    <atom:link href="https://aiforthecurious.ai/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <link>https://aiforthecurious.ai/feed/price-the-correct-task</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/price-the-correct-task</guid>
      <description><![CDATA[<p>When an AI agent costs too much to run, the obvious fix is a cheaper model. The model’s usage charges are the cost you can see. <em>But the people checking the agent’s work can cost much more than the model does.</em></p>

<p>Say you run an agent (an AI system that does a task start to finish on its own) that processes supplier invoices. The model provider charges you by usage, roughly by how much text the model reads and writes. So a cheaper model looks like an easy saving.</p>
<p>But the agent can finish an invoice and still get it wrong. It reads the wrong amount, picks the wrong supplier, or tries to post the same payment twice. Someone has to catch that, redo the invoice and fix whatever already went through. That work goes up with how often the agent is wrong and how many invoices it handles, and comparing usage charges between models doesn’t see any of it.</p>
<p>McKinsey put numbers on this in 2026, for one bank’s customer-service workflow. Of the running costs that grow with each request handled, the model’s usage charges were about 20-25%. The people checking and correcting its work were about 70-75%. That’s one bank’s workflow, not a rule for every system.</p>
<p>In your system the usage charges may really be the bigger cost. That can happen when the agent rarely gets things wrong and little of its work needs checking. To find out, I’d count how many invoices a person had to check, redo or fix, and what that time cost, then put it next to the usage charges.</p>
<p>A pricier model that gets it right the first time can come out cheaper than a bargain one that keeps needing fixes. You’ll only see that if your system records whether each invoice was actually processed correctly, not just whether the agent finished running.</p>
<p>So before you switch to a cheaper model, work out what it costs to get one invoice done correctly, retries and human checking included.</p>]]></description>
      <pubDate>Mon, 09 Nov 2026 11:21:55 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/new-video-limit-what-the-agent-can-do</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/new-video-limit-what-the-agent-can-do</guid>
      <description><![CDATA[<p>Why can’t a strict rule in the prompt stop prompt injection? A short explainer for engineers building AI agents: where the boundary actually goes, and two tests for what an agent should do on its own. <a href="https://aiforthecurious.ai/videos/limit-what-the-agent-can-do" class="internal internal-link alias" data-slug="videos/limit-what-the-agent-can-do">Watch the video</a>.</p><figure><a href="https://aiforthecurious.ai/videos/limit-what-the-agent-can-do"><img src="https://aiforthecurious.ai/videos/media/limit-what-the-agent-can-do-poster.png" alt="Video poster: What is prompt injection, and why can't prompts stop it?"/></a></figure>]]></description>
      <pubDate>Sun, 08 Nov 2026 11:53:27 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/put-a-cheaper-model-first</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/put-a-cheaper-model-first</guid>
      <description><![CDATA[<p>Say you’re paying an LLM to route every support ticket to the right team. You could put a cheaper model in front of it, let that model handle the tickets it’s confident about, and send only the rest to the LLM. <em>That only works if “confident” means “correct”.</em></p>

<p>I’ve been reading about one such model, Jev, which TypeSafe AI released in early access in September. It doesn’t write text. You give it a ticket and a list of teams, and it returns a probability for each team. TypeSafe pitches it as much cheaper than an LLM, though that’s its own claim.</p>
<p>A customer writes “I was charged twice and need this fixed.” The model comes back with something like <code>billing: 0.94</code>, its probability that this ticket belongs to billing. That’s confident enough to clear the threshold you set, so plain code sends the ticket to billing. A ticket where the top probability is lower goes on to the LLM or a person. Say 80 of every 100 tickets clear the threshold (my number, just to show the shape). The cheaper model still runs on all 100, but your LLM only sees 20.</p>
<p>So the cheaper model doesn’t have to be as good as the LLM on everything, because the hard tickets still reach the LLM. What I’d look at first is what happens when it’s wrong. A low probability costs you an extra LLM call. <em>A wrong answer at 0.94 goes straight to code, and nothing in the pipeline checks it until someone downstream notices.</em></p>
<p>I’d want to know whether those probabilities are calibrated: when the model says 0.9, is it right about 90% of the time? I don’t know yet whether Jev’s probabilities are calibrated, so I’d test that on real tickets before trusting the threshold.</p>
<p>That’s a check before you launch. Once it’s live, I’d have someone review a random sample of the tickets it routed on its own. That won’t catch every mistake, but it tells you how many are getting through.</p>
<p>You’ll hear that two models means two things to monitor. True, and that’s why I’d measure the whole pipeline instead of the cheaper model alone:</p>
<ul>
<li><strong>Mistakes that get through:</strong> wrong answers the cheaper model acted on without passing them to the LLM or a person, counted from that sample.</li>
<li><strong>Escalation rate:</strong> if most tickets get passed up anyway, you’re paying for both models on most of them and the savings shrink.</li>
<li><strong>End-to-end cost and delay:</strong> including the hard tickets, which now wait for both models.</li>
</ul>
<p>So before you put a cheaper model in front of your LLM, decide how you’ll catch its confident mistakes.</p>]]></description>
      <pubDate>Wed, 04 Nov 2026 12:10:38 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/high-acceptance-isnt-high-quality</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/high-acceptance-isnt-high-quality</guid>
      <description><![CDATA[<p>When people accept 96% of an AI’s suggestions, it’s easy to assume the AI is working. <em>That number can mean the AI became extremely good. It can also mean people stopped checking.</em></p>

<p>The rate looks the same either way, because it counts what people approved, not whether it was right. And when people review similar AI recommendations over and over, they may start trusting them automatically. So the rate can go up while the checking goes down.</p>
<p>I’d test the reviewers directly. Put some AI output you already know is wrong into their queue and see how many they catch. That gives you the false acceptance rate (the share of bad output people approved), the number I’d care about most. If it’s high, the 96% isn’t telling you much about the AI.</p>
<p>You’d need enough planted items to trust the number, and they need to look like real mistakes, or you’ll only learn who can spot a test. And since someone may approve one, your system has to catch it after the review, before it actually goes out. Even then, I’d trust a bad score more than a good one.</p>
<p>The review time is small, because people are already working through the queue. The real work is preparing the bad examples and building that catch. I’d also sample what got accepted, to see how much bad output gets through, and size that sample to how much a bad approval would hurt.</p>
<p>I’m not saying high acceptance is bad. The AI may really be that good. It’s just ambiguous without a quality measure.</p>
<p>So before you read a high acceptance rate as good news, plant some known mistakes and sample what got accepted.</p>]]></description>
      <pubDate>Mon, 02 Nov 2026 12:35:28 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/not-every-agent-step-needs-an-llm</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/not-every-agent-step-needs-an-llm</guid>
      <description><![CDATA[<p>An AI agent is a language model in a loop, so it’s easy to assume every step in it needs one. <em>But look at what those steps ask: many aren’t asking the model to write anything, just to pick.</em></p>

<p>Take a simple agent loop. It works out what the user wants. It chooses the next action: search the database, call an API, ask the user, or stop. Then it checks the result and either continues or retries.</p>
<p>The second and third steps end in a pick from a short list you could write down in advance. So do a lot of the other questions agents send to an LLM: is this document relevant, did that tool call succeed, which specialist agent should take this task. Those are classification and routing problems (sorting an input into one of a few known answers).</p>
<p>You can still use an LLM for them, and it may work well. But then you’re using a model built to write as a classifier, and it can cost more and take longer than a pick needs, especially if it reasons through the question before it answers.</p>
<p>That’s the argument behind Jev, which TypeSafe AI released in early access in September: a model that only decides. You give it the state and the options, and it returns probabilities over those options instead of text.</p>
<p>The LLM may well stay on some of those steps. The difference is that you picked it for a decision on purpose. One boundary I’d keep in mind: a short answer isn’t the same as an easy question. “Is this document relevant?” is yes or no, but getting it right can take real understanding. Whatever you hand a step to, test it on that step.</p>
<p>So go through your agent workflow and mark every step whose answer comes from a list you could write down.</p>]]></description>
      <pubDate>Mon, 26 Oct 2026 11:04:18 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/new-video-judge-ai-on-your-own-workflow</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/new-video-judge-ai-on-your-own-workflow</guid>
      <description><![CDATA[<p>A draft that appears in 5 seconds isn’t a ticket done in 5 seconds. In my new video, I walk through why AI often moves work rather than removes it, and how to judge it by your whole workflow instead of the fast step. <a href="https://aiforthecurious.ai/videos/judge-ai-on-your-own-workflow" class="internal internal-link alias" data-slug="videos/judge-ai-on-your-own-workflow">Watch the video</a>.</p><figure><a href="https://aiforthecurious.ai/videos/judge-ai-on-your-own-workflow"><img src="https://aiforthecurious.ai/videos/media/judge-ai-on-your-own-workflow-poster.png" alt="Video poster: Judge AI By The Whole Workflow, Not The Fast Step"/></a></figure>]]></description>
      <pubDate>Sun, 25 Oct 2026 11:44:01 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/approve-the-action-not-the-reasoning</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/approve-the-action-not-the-reasoning</guid>
      <description><![CDATA[<p>If your team plans to trust an AI agent by reading its reasoning, I’d look somewhere else first: at what the agent actually does.</p>

<p>The reasoning is the agent’s plan. <em>The tool call it actually makes is what changes your systems</em>, and the two can differ. Once an agent can modify code, update a database, or trigger a business workflow, the action is the part with consequences, so that’s what you review and audit.</p>
<p>In practice, I’d put the checks at the boundary, not inside the model: validate each tool call before it runs, and turn every state change into a structured record someone can approve and audit. I’d still keep the reasoning for debugging why an action went wrong - it just isn’t what you approve.</p>
<p>This matters more now because reading the reasoning may stop being an option. A recent research preprint describes HybridAgent, a model that does part of its reasoning as numbers inside the model instead of words. The researchers say that reasoning can be translated back into rough words, but with a hosted model you’d see that only if the provider chose to show it.</p>
<p>So if your agent can change things, put the approval check on the action record, not on the reasoning trace.</p>]]></description>
      <pubDate>Sat, 24 Oct 2026 13:58:21 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/measure-reasoning-tokens-per-task</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/measure-reasoning-tokens-per-task</guid>
      <description><![CDATA[<p>An AI agent that reasons in text writes out every step as words, including the steps only it will read.</p>
<p>Those text outputs do two jobs. Some are the model’s scratch work, read only by the model. Some are messages to a tool or a person. But theoretically speaking, only the messages have to be words; the scratch work could stay as numbers inside the model.</p>

<p>Generating tokens/words as outputs is computationally expensive, because a model produces text one token at a time and each token takes a full pass through the model. On today’s reasoning models, those tokens are billed as output, even when the API never shows them to you. Think of it like a program that prints every intermediate calculation to the screen and reads it back in.</p>
<p>HybridAgent, from a new research paper, splits the two jobs: the model would reason in vectors (the numerical values it computes internally) and switch to text only to call a tool or give an answer to the user. So it keeps that work in RAM and writes out only for an API call or the final result.</p>
<p>But here’s the tradeoff: <em>reasoning that never becomes words is harder for a person to read.</em></p>
<p>So if you’re building an agentic workflow using a reasoning model, you might want to check how many of the output tokens you pay for are the model’s own reasoning. You can do that by:</p>
<ol>
<li>Logging the visible output tokens and the reasoning tokens as two separate numbers for every model call in the loop. Every major API reports the reasoning count in its usage data.</li>
<li>Sum both per task, not per call, as an agent makes many calls per task.</li>
<li>Compare the two. If most of what you pay for per task is reasoning, that’s where the lower-effort setting or a cap on reasoning tokens might cut costs.</li>
</ol>]]></description>
      <pubDate>Wed, 21 Oct 2026 10:35:56 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/where-the-model-runs</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/where-the-model-runs</guid>
      <description><![CDATA[<p>Adding an AI feature to a product usually starts with picking a vendor’s API.</p>
<p>That one choice decides two things: which model you use, and where it runs. With a closed model like GPT or Claude, the vendor or one of its cloud partners runs it. You can often pick a region, but your data still leaves your own systems.</p>

<p>Open-weight models separate those decisions. The trained model is published, so the same model can run in several places. A cloud provider can host it for you, you can run it on your own GPUs. You can also further train a copy on your own examples (fine-tuning). Each option trades off cost, speed, security, who controls the data, quality and upkeep differently.</p>
<p>That matters most for a feature over data whose location is regulated. If the rules say the data can’t leave your systems, that can rule out options before you ever compare benchmarks.</p>
<p>This assumes an open model is good enough for the task and your team can operate one. Self-hosting isn’t automatically better.</p>
<p>So before you compare models, find out where your data is allowed to go. The answer may rule out some models and hosting options before you start.</p>]]></description>
      <pubDate>Mon, 19 Oct 2026 10:24:18 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/compute-bill-vs-memory-bill</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/compute-bill-vs-memory-bill</guid>
      <description><![CDATA[<p>In my last post, I wrote about Beam, the open model from Reflection AI. One number in its announcement is easy to misread: <em>it uses 23B of its 501B parameters for each token it processes.</em></p>
<p>For those not familiar with this, parameters are the numbers a model learns in training. These billions or even trillions numbers determine how the model processes inputs and generates outputs (tokens). A token is a word or piece of a word. To run the model, they have to be loaded into GPU memory.</p>
<p>Reflection AI claims that Beam needs less computing power to run comparing to other models due to the nature of how it’s built.</p>

<p>In a traditional “dense” AI model, every single part of the neural network activates for every prompt, regardless of whether you are asking it to write a poem or solve a complex math equation.</p>
<p>Beam uses a “mixture-of-experts” (MoE) architecture where it’s divided into several smaller, specialized sub-networks, called experts. When you give it a prompt, a “router” quickly analyzes your request and sends it only to a few experts best suited to handle that specific topic. The rest stay inactive. By only lighting up a fraction of its total “brain” for any given task, an MoE model can have the vast knowledge capacity of a massive model but operate with the speed and computational lightness of a much smaller one.</p>
<p>So the math <em>per token</em> is roughly a 23B model’s. But the router chooses again for the next token, and it can pick any expert, so all 501B still have to stay loaded.</p>
<p>The 23B is your <em>compute bill.</em> The 501B is your <em>memory bill.</em> Reflection AI’s claim, which is based partly on estimates rather than measured serving cost, is about the first one - the <em>compute cost.</em></p>
<p>With that being said, you can shrink the memory cost by storing each number with fewer bits (<em>quantizing</em>), or keep idle experts in slower memory and pull them in when needed (<em>offloading</em>). The first costs some <em>precision</em> and the second costs <em>speed.</em></p>
<p>So if you’re estimating hardware from a spec sheet that leads with “active parameters,” don’t mistake that number for the model’s size. It’s the “computing per token.” Make sure to size your GPU memory from the total.</p>]]></description>
      <pubDate>Sat, 17 Oct 2026 12:24:40 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/why-nvidia-wants-open-models</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/why-nvidia-wants-open-models</guid>
      <description><![CDATA[<p>When the company that sells most AI chips looks at buying an AI lab, it reads like a move to compete with OpenAI.</p>
<p>Reuters, citing the Financial Times, reports Nvidia is in talks to invest further in Reflection AI. Reflection builds open-weight models - trained models that are published so anyone can download and run on their own hardware. In fact, it just announced a new open-weight model called Beam.</p>

<p>Nvidia hasn’t said why it’s interested. But let’s compare the following two futures:</p>
<p>In one, three companies (OpenAI, Anthropic, Google) supply nearly all the AI models, and everyone else pays to use them. Nvidia’s chips mostly go to those big three and their cloud partners.</p>
<p>In the other, open models get good enough that banks, hospitals, and software companies run their own. Now thousands of organizations need chips.</p>
<p>Nvidia doesn’t need to own the winning model. It only needs more organizations running models.</p>
<p>That only works if two things hold:</p>
<ul>
<li>Demand for computing has to grow faster than models get cheaper to run.</li>
<li>And those organizations have to buy Nvidia’s chips rather than AMD’s or Google’s.</li>
</ul>
<p>The second part is where Nvidia’s software comes in. An open model can run on anyone’s chips, but Nvidia also builds the tools that tune models to run well on its hardware. Working closely with a lab like Reflection could make that lab’s model faster or cheaper to run on Nvidia’s hardware.</p>
<p>AI infrastructure has layers: <em>apps on top, then models, then the software that runs them, then chips.</em></p>
<p>Nvidia’s advantage has mostly been in the bottom two layers. But those layers depend on each other:</p>
<ul>
<li>How a model is built decides how much computing it needs</li>
<li>Software decides how efficiently that computing gets done</li>
<li>And how widely the apps get used decides how much computing is needed in total</li>
</ul>
<p>By helping build open models, Nvidia can influence all three.</p>]]></description>
      <pubDate>Thu, 15 Oct 2026 13:36:20 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/diagnose-before-upgrading-the-model</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/diagnose-before-upgrading-the-model</guid>
      <description><![CDATA[<p>“The PoC for an agentic AI implementation isn’t delivering value. Let’s try a better model.”</p>
<p>That recommendation already assumes a diagnosis we haven’t established.</p>
<p>Reaching for a stronger model assumes model capability is what’s preventing the current implementation from delivering the expected results. It might be. But that’s still a hypothesis, which should be tested before it gets funded.</p>

<p>Here’s a concrete case:
A team runs a PoC where AI drafts customer-support replies. Drafting drops from 5 minutes to 5 seconds. But human support agents then spend 2 minutes checking the draft reply and 3 minutes rewriting it. End to end, a ticket takes about as long as it did before.</p>
<p>We know the time savings disappeared into review and rewriting. We don’t yet know why the human support agents have to do that work.</p>
<p>The answer changes what you try next.</p>
<p>If agents are rewriting drafts to add customer account history the AI never received, a better model probably won’t help much. It still won’t have the account history. What might help is giving it the context that it needs in the first place.</p>
<p>If the model already has the context and keeps using it incorrectly, testing a more capable model may be justified.</p>
<p>The symptom is the same in both cases, but the fix is different. Knowing the cause tells you whether to spend the next iteration on context or on the model. Without it, a model upgrade is just the most expensive guess.</p>
<p>So before choosing the next model, name the problem you expect it to solve, and the evidence that points there.</p>]]></description>
      <pubDate>Tue, 13 Oct 2026 09:57:32 GMT</pubDate>
    </item>
    <item>
      <title>AI Made Your Team Faster. Why Are Reviews Slower?</title>
      <link>https://aiforthecurious.ai/feed/understand-the-code-you-submit</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/understand-the-code-you-submit</guid>
      <description><![CDATA[<p>Your teammate builds a feature with AI in 20 minutes, and you spend the afternoon working out what it does. If you’ve started to wonder whether you’re the bottleneck, here’s how I think about it: the review burden usually has two different sources, and they need two different fixes.</p>
<p>This video separates the amount of code coming at reviewers from the understanding that hasn’t happened before the code reaches them. It uses one example (a spreadsheet import that quietly brings back an old phone number) to show what understanding a change actually involves, and ends with two things to try on your next few pull requests.</p>
<figure><a href="https://aiforthecurious.ai/feed/understand-the-code-you-submit"><img src="https://aiforthecurious.ai/feed/media/fast-to-understand-thumbnail.png" alt="Video poster"/></a></figure><p><a href="https://aiforthecurious.ai/feed/understand-the-code-you-submit">Watch the video (10:54)</a></p>
<h2>Transcript</h2>

<h3 id="0000-am-i-the-bottleneck">[00:00] Am I the bottleneck?<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0000-am-i-the-bottleneck" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Your teammate uses AI to build a feature in 20 minutes. The code looks tidy. There are tests. There’s even a detailed pull request description. Beautiful! Then you start reviewing it.</p>
<p>You ask what happens in a particular situation. The author isn’t sure, so you trace the implementation, work out what it does, discover a problem, and explain it back to them. And by the time you finish, another PR has arrived.</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen A pull request mock-up: &quot;#482 Import customers from spreadsheet&quot;, built in 20 minutes, 12 tests passing. The review thread follows, and a second PR lands on top of the first. </p></div>
                  
                </div>
</blockquote>
<p>If that sounds familiar, you might be wondering, “Am I becoming the bottleneck for the team?” I think that question hides two different problems. One is the amount of code coming at you. The other is the amount of understanding that hasn’t happened before the code reaches you. And addressing one won’t necessarily fix the other.</p>
<h3 id="0050-two-sources-of-review-burden">[00:50] Two sources of review burden<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0050-two-sources-of-review-burden" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>So let’s start with the volume. If your team produces changes faster than people can review them, the queue grows. Even well-written code from capable developers still requires someone’s attention.</p>
<p>Then there’s a second problem. An author submits an implementation they don’t understand well enough to explain or evaluate. Now the reviewer has to figure out how it works, assess whether it’s appropriate, and bring the author up to speed. Some of the work that appeared to be finished is still waiting for someone to do it.</p>
<p>Research suggests that experience still affects how developers work with AI tools. In one study, researchers analyzed about 23,000 pull requests created with AI assistance. They found that pull requests from less-experienced developers received more review comments and took longer to resolve than those from more-experienced developers.</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen The study: <a href="https://arxiv.org/abs/2602.23905" class="external external-link">Novice Developers Produce Larger Review Overhead for Project Maintainers while Vibe Coding<svg aria-hidden="true" class="external-icon" style="max-width:0.8em;max-height:0.8em;" viewBox="0 0 512 512"><path d="M320 0H288V64h32 82.7L201.4 265.4 178.7 288 224 333.3l22.6-22.6L448 109.3V192v32h64V192 32 0H480 320zM32 32H0V64 480v32H32 456h32V480 352 320H424v32 96H64V96h96 32V32H160 32z"></path></svg></a> (Asdaque et al., arXiv, February 2026). 22,953 pull requests from 1,719 developers; pull requests from less-experienced developers received 4.52× more review comments and stayed open 5.16× longer before resolution.</p></div>
                  
                </div>
</blockquote>
<p>Now there is a caveat to this that I should call out though. More comments don’t necessarily prove the code was worse, and the study doesn’t establish that AI caused the additional review overhead. In other words, the study compared less-experienced developers using AI against more-experienced developers using AI. They did not establish how those same experience groups would have performed without AI. But it gives us a reason to question the idea that producing an implementation means the difficult work is over.</p>
<h3 id="0227-what-understanding-a-code-change-involves">[02:27] What understanding a code change involves<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0227-what-understanding-a-code-change-involves" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>So what does understanding a code change actually involve? Imagine your team needs a feature that imports customer information from a spreadsheet. A new customer should be created, and an existing customer should be updated. A developer uses AI to build it. They try a file, the customers appear in the database. They try updating an existing customer, and that works too. The tests pass, and they submit the change for review. So far, everything looks reasonable.</p>
<p>Then you ask, “What happens if somebody imports a spreadsheet they exported six months ago?” Let’s say a customer recently changed their phone number. Your database has the new number, but the spreadsheet still has the old one. The import finds the matching customer and replaces their phone number with the value from the spreadsheet.</p>
<p>The old number is back, nothing crashed, and the test still passes, but the test checks that the importer replaces the existing value with the spreadsheet values. That tells us the update logic behaves as the test expects. It doesn’t establish whether overwriting new information is what we want.</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen Customer 1042: the database has the phone number changed last week (555-0199); the six-month-old spreadsheet still has 555-0142. After the import, the database shows 555-0142 again. The test that still passes: <code>expect(customer.phone).toBe(row.phone)</code>.</p></div>
                  
                </div>
</blockquote>
<p>And maybe it is. Maybe the business treats the spreadsheet as the authoritative source. Or maybe the import should flag conflicting information for someone to check. The point is, we need to establish that before deciding whether this behavior is acceptable. That is part of what the review process needs to examine: how the code behaves in situations that matter, and whether that matches the outcome we intended.</p>
<h3 id="0404-when-the-author-understands-the-code">[04:04] When the author understands the code<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0404-when-the-author-understands-the-code" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Now the author hadn’t considered this scenario at all. That’s normal as reviewers sometimes bring another perspective, they discover things the author missed. But watch what happened next.</p>
<p>Imagine the author says, “I didn’t think of that. I checked that the import worked. But I’m not sure how it decides which customer to update or which fields it replaces.” So you as the author start tracing the implementation. You find the matching logic, follow it into the update function, and check whether anything protects the existing information. Then you explain it back to the author. You’re now doing two jobs: reviewing the change and helping the author understand the implementation they submitted.</p>
<p>Now let’s replay that scenario with an author who understands the code. So the same implementation, same tests, same missed case. The author may say something like: “I didn’t think of an old export scenario either. We match customer by email and copy the imported fields onto the existing record. But let me check whether anything in that path compare how recent the information is.” They go back, they trace the path, and they come back with: “There’s no freshness checks. I reproduced it. The older number replaces the current one. Our test checks that the values get replaced, but it never tries stale data. So we need to establish which source should take precedence.”</p>
<p>In this case, the reviewer still discovered the problem. But the difference is that the author has something to learn. They investigated the concern themselves. They were able to explain the consequence and resolve it.</p>
<p>That’s what I mean by understanding the code you submit. You don’t have to anticipate every question the reviewer might ask or remember every line of code. But as the author of the code, you should be able to work through the important behavior and check your explanation against the implementation.</p>
<p>LLVM, a widely used open-source project offering a collection of tools and libraries for building and optimizing software, makes a similar expectation explicit in its AI contribution policy. Contributors must review generated material before submitting it and be able to answer questions about their work.</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen <a href="https://llvm.org/docs/AIToolPolicy.html" class="external external-link">LLVM's AI Tool Use Policy<svg aria-hidden="true" class="external-icon" style="max-width:0.8em;max-height:0.8em;" viewBox="0 0 512 512"><path d="M320 0H288V64h32 82.7L201.4 265.4 178.7 288 224 333.3l22.6-22.6L448 109.3V192v32h64V192 32 0H480 320zM32 32H0V64 480v32H32 456h32V480 352 320H424v32 96H64V96h96 32V32H160 32z"></path></svg></a>, with its two relevant sentences highlighted: contributors must read and review all LLM-generated code or text before asking others to review it, and should be &quot;able to answer questions about their work during review.&quot;</p></div>
                  
                </div>
</blockquote>
<h3 id="0618-making-the-volume-manageable">[06:18] Making the volume manageable<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0618-making-the-volume-manageable" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>That helps with the second problem: understanding being passed downstream. But we still have the first problem: the volume.</p>
<p>Suppose the import PR also reorganizes the customer service, renames several classes, and adds a reusable framework for future imports. The reviewer now has several decisions to evaluate together.</p>
<p>Before submitting, ask yourself, which changes actually belong together? Could the unrelated refactor be separate? Do we need that additional framework for the current feature? The goal is a change with a clear purpose that someone can follow. For example, keep the changes needed for the import together, including its tests, separate unrelated refactoring, and leave out conditions the feature doesn’t require.</p>
<p>A line limit can warn you that a PR is getting large. But even a short PR can mix unrelated changes, so keep that in mind. So, the useful question is, can the reviewer follow one clear purpose through this change? Google’s engineering guidance actually recommends small, focused changes and generally separating refactors from feature changes.</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen Google's engineering practices, <a href="https://google.github.io/eng-practices/review/developer/small-cls.html" class="external external-link">Small CLs<svg aria-hidden="true" class="external-icon" style="max-width:0.8em;max-height:0.8em;" viewBox="0 0 512 512"><path d="M320 0H288V64h32 82.7L201.4 265.4 178.7 288 224 333.3l22.6-22.6L448 109.3V192v32h64V192 32 0H480 320zM32 32H0V64 480v32H32 456h32V480 352 320H424v32 96H64V96h96 32V32H160 32z"></path></svg></a>: the right size for a change is one self-contained change, and refactorings usually belong in a separate change from feature work or bug fixes.</p></div>
                  
                </div>
</blockquote>
<p>And if the review queue keeps growing, the team needs to make room to finish that work. That might mean helping with reviews, responding to feedback, or removing whatever is blocking a change before starting another one. Generating more code looks like progress. But when work is piling up in a review, it’s easy to miss that review is where things are stuck.</p>
<h3 id="0752-using-ai-to-build-understanding">[07:52] Using AI to build understanding<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0752-using-ai-to-build-understanding" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>So there’s an obvious question here, couldn’t we use AI to explain the code or review it too? Yes, but a clear explanation can feel convincing before you even check whether it’s correct.</p>
<p>So here’s a simple approach I would try. Take one important behavior in the change, before asking AI to explain it, write out what you think will happen. For this CSV importer example, we can ask: if this customer already exists, which fields will change? Then use AI to help locate the relevant code and trace the path. Compare that path with your prediction. If they differ, find out why. Finally, run a small example or check a meaningful test. Does the observed behavior agree with your explanation?</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen The four steps: Predict, Trace with AI, Compare, Check. </p></div>
                  
                </div>
</blockquote>
<p>You can also ask AI to suggest a situation you haven’t considered, then investigate it yourself, using the code and tests as evidence.</p>
<p>In a small study from Anthropic, participants who use AI to ask conceptual questions or understand generated code tended to score higher on a follow-up comprehension quiz than those who relied on AI to complete the coding or fix errors for them.</p>
<blockquote class="callout visual" data-callout="visual">
<div class="callout-title">
                  <div class="callout-icon"></div>
                  <div class="callout-title-inner"><p>On screen Anthropic, <a href="https://www.anthropic.com/research/AI-assistance-coding-skills" class="external external-link">How AI assistance impacts the formation of coding skills<svg aria-hidden="true" class="external-icon" style="max-width:0.8em;max-height:0.8em;" viewBox="0 0 512 512"><path d="M320 0H288V64h32 82.7L201.4 265.4 178.7 288 224 333.3l22.6-22.6L448 109.3V192v32h64V192 32 0H480 320zM32 32H0V64 480v32H32 456h32V480 352 320H424v32 96H64V96h96 32V32H160 32z"></path></svg></a> (January 2026), a study with 52 developers learning a new Python library. Participants who used AI to build understanding averaged 65% or more on the quiz; those who relied on it to write or debug code averaged under 40%.</p></div>
                  
                </div>
</blockquote>
<p>Now this doesn’t prove that this particular practice will make your review faster. But it does suggest that how we use AI deserves attention, and not just whether we should use it.</p>
<h3 id="0917-what-to-try-on-your-next-pr">[09:17] What to try on your next PR<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0917-what-to-try-on-your-next-pr" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Now, I acknowledge that all of this takes time. If authors spend longer on their pull requests, does that reduce repeated investigation and rework? Does useful work reach completion sooner? Does quality improve? A shorter review by itself isn’t enough to answer those questions. But the goal is to arrive at review able to help evaluate and improve your own code without that burden falling on the reviewer entirely.</p>
<p>So for your next few PRs, I would try two things. As an author, make sure you can explain and evaluate the important behavior you’re submitting. As a team, keep changes focused and make room to review the work already in progress.</p>
<p>Reviewers will still discover things authors missed. That’s valuable. The goal is for the author to be able to help investigate those discoveries, and for the reviewer to have a manageable change to review.</p>
<p>AI can shorten the path to an implementation. But understanding that implementation is still part of the definition of done.</p>
<p>So, if you’ve been wondering whether you are the bottleneck for the team, look at what’s actually taking your time. If it’s the amount of code, that’s a team problem. If it’s understanding that should have happened before the code reaches you, that’s a different problem, with a different fix.</p>
<p>And for the authors, before you submit your next pull request, ask yourself this: “Could I help someone reason through what this code will do? Or would they have to explain it to me first?”</p>]]></description>
      <pubDate>Mon, 12 Oct 2026 10:58:43 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/what-revokes-an-agents-access</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/what-revokes-an-agents-access</guid>
      <description><![CDATA[<p>An abandoned AI project can leave behind something you probably don’t want: <em>an identity that still has access to your production systems.</em></p>
<p>Imagine a team experimenting with an internal AI agent that helps support engineers investigate customer issues. They give it a service identity with read access to production customer data.</p>
<p>The prototype works, but priorities change. The project gets abandoned, and everyone moves on.</p>

<p>Six months later, the service identity still exists, with the same production permissions. If its credentials leaked through an old CI/CD pipeline or a compromised development environment, an attacker could use them to read customer data. From the system’s perspective, those requests come from a legitimate identity with valid permissions. And because nobody is actively maintaining that project anymore, who is monitoring that identity’s activity? Who is reviewing whether it still needs access?</p>
<p>Now multiply that across dozens of teams experimenting with internal AI agents, each creating identities and granting permissions to different systems. You could end up with hundreds of identities that nobody uses or reviews, all still holding valid permissions.</p>
<p>This made me realize there’s a hidden assumption in how we manage identity lifecycles: <em>something will tell us when an identity is no longer needed.</em></p>
<p>For employees, we have HR triggers. Someone leaves the company, and an offboarding process revokes their access. An agent doesn’t resign. Its project might simply get abandoned. So what triggers that process for an agent?</p>
<p>This isn’t a new security problem. We’ve dealt with forgotten service accounts for years. But as organizations experiment with more internal AI agents, I think we’re going to see more of it unless we design for it.</p>
<p>A few things I’d want in place:</p>
<ul>
<li>Every agent identity has a designated owner accountable for its access.</li>
<li>Temporary identities expire unless someone explicitly renews their authorization.</li>
<li>Retiring a project automatically triggers revocation of its associated identities and permissions.</li>
</ul>
<p>Short-lived access tokens help, but they’re not sufficient if the underlying identity can keep obtaining new tokens indefinitely.</p>
<p>So when reviewing the architecture of an internal AI agent, I’d add one more questions:</p>
<p><em>“What will cause this agent’s access to be revoked when nobody needs it anymore?”</em></p>]]></description>
      <pubDate>Thu, 08 Oct 2026 10:33:10 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/where-human-judgment-adds-value</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/where-human-judgment-adds-value</guid>
      <description><![CDATA[<p>“Human in the loop” sounds like a safety strategy. But I’m starting to think it’s too vague to be useful.</p>

<p>Imagine a coding agent makes 20 changes:</p>
<ul>
<li>formats a file</li>
<li>adds test scaffolding</li>
<li>updates documentation</li>
<li>refactors some repetitive code</li>
<li>changes an authorization rule</li>
<li>modifies a database schema</li>
</ul>
<p>If a human has to carefully approve every one of those changes, we’ve created a review bottleneck and may have erased much of the productivity gain.</p>
<p>But letting the agent make all the decisions autonomously doesn’t seem right either. I’m starting to see human-in-the-loop less as a review strategy and more as a risk-allocation decision.</p>
<p>I’d put more human judgement around a change when:</p>
<ul>
<li>getting it wrong has meaningful consequences,</li>
<li>the change is difficult to reverse,</li>
<li>or deciding what’s correct depends on context the AI may not have.</li>
</ul>
<p>Most decisions then land in one of the following zones:</p>
<ul>
<li><strong>Low stakes, clear tasks:</strong> let the agent act. For example, code formatting, boilerplate, test scaffolding, mechanical refactors, docs.</li>
<li><strong>Low stakes, fuzzy tasks:</strong> let the agent propose, and spot-check a sample instead of reviewing everything.</li>
<li><strong>High stakes:</strong> a human approves at minimum. Auth, schema migrations, concurrency, public API contracts, destructive operations. If the requirements are also unclear, the human makes the decision instead of just signing off on the agent’s.</li>
</ul>
<p>Even “high consequence” doesn’t always mean a human must inspect every individual action. Sometimes the better answer is a deterministic control.</p>
<p>For example, instead of asking someone to review every AI-generated refund, the system might simply enforce a rule: “Refunds over $500 require human approval.”</p>
<p>The goal isn’t to put a human between the AI and everything it does. It’s to decide where human judgement actually adds value, and where software can enforce the boundary instead.</p>]]></description>
      <pubDate>Wed, 07 Oct 2026 10:31:56 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/faster-coding-isnt-faster-delivery</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/faster-coding-isnt-faster-delivery</guid>
      <description><![CDATA[<p>I used to think the productivity gain from AI was pretty straightforward:</p>
<p>If coding takes 25 minutes with AI instead of 2 hours without it, that’s a big productivity improvement.</p>
<p>But I’m realizing that’s measuring the wrong boundary.</p>

<p>Workday published a breakdown that shows this well. Imagine an engineer’s work normally looks like this:</p>
<p>Implementation: 120 min<br/>
Testing: 30 min<br/>
Code review: 30 min<br/>
Rework: 20 min</p>
<p>Now give them an AI coding agent and implementation drops to 25 minutes. Great!</p>
<p>But inspecting the generated code takes 35 minutes. Code review increases to 45 and rework increases to 40.</p>
<p>The implementation step became almost 80% faster, but the end-to-end workflow only improved by about 11%.</p>
<p>AI didn’t remove all that work. Some of it moved downstream into verification, review, and rework.</p>
<p>So now I don’t ask how much faster the task became with AI. I ask how much faster “useful work” moved through the whole workflow, without more defects, rework, or incidents.</p>
<p>I’m not saying that speed doesn’t matter. I’m saying that <em>local speed</em> is only meaningful if it produces an <em>end-to-end improvement</em>.</p>]]></description>
      <pubDate>Mon, 05 Oct 2026 15:07:45 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/ai-defaults-are-unmade-security-decisions</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/ai-defaults-are-unmade-security-decisions</guid>
      <description><![CDATA[<p>AI-generated code can pass every automated security check you give it and still contain a security decision nobody actually made.</p>
<p>Imagine an AI coding assistant builds authentication for a fintech app and gives session tokens a 7-day lifetime. A static code analyzer might see:</p>
<ul>
<li>the token is generated correctly</li>
<li>the signing algorithm is approved</li>
<li>the secret isn’t hard-coded</li>
<li>no known vulnerability is present</li>
</ul>
<p>Everything passes. ✅</p>
<blockquote>
<p><em>But should a session that can authorize financial transactions stay valid for 7 days?</em></p>
</blockquote>

<p>It depends on what else protects it.</p>
<ul>
<li><em>Does moving money require re-authentication, no matter how old the session is?</em> If it does, a stolen session can look at the account but can’t drain it, so a long lifetime is less dangerous.</li>
<li><em>Can a stolen session be revoked immediately?</em> If the server can kill a session the moment fraud is detected, the risk is somewhat mitigated.</li>
<li><em>Is the token tied to the user’s device?</em> If the token only works on the phone it was issued to, someone who copies it to another device can’t use it.</li>
</ul>
<p>If someone had already decided “re-authentication after 15 minutes is required” and encoded this rule in the code analyzer, the scanner could enforce it. But if nobody has made that decision, the scanner has nothing to check against.</p>
<p>Security automation is good at enforcing boundaries we’ve already defined. It can’t tell us where a new boundary should be.</p>
<p>That was true before AI. But when you wrote the code yourself, you at least had to pick the value. When AI writes your code, it could pick a “reasonable-looking” default, and it’s easy to merge it without noticing.</p>
<p>For security-sensitive behavior, someone still has to understand what the system can do and decide what should be allowed. Then, wherever possible, turn that decision into a rule the pipeline enforces so the next default gets flagged automatically by the code analyzer.</p>
<p>For platform teams, consider building a shared auth library with the defined rules and enforcing it by adding a pipeline check that flags code overriding it.</p>]]></description>
      <pubDate>Sat, 03 Oct 2026 11:50:38 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/least-privilege-isnt-enough-for-agents</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/least-privilege-isnt-enough-for-agents</guid>
      <description><![CDATA[<p>Least privilege answers an important security question:</p>
<p><em>What is this identity allowed to do?</em></p>
<p>For AI agents, that still matters. But it’s no longer the whole question.</p>

<p>If an agent has permission to issue a refund, update a customer record, or send an email, we should assume that capability may eventually be exercised. That was true for traditional software too.</p>
<p>What changes with an agent is the decision-making system inside that permission boundary.</p>
<p>Imagine a customer-support agent with this permission: <code>refund:create</code>.</p>
<p>When it was originally approved, the agent used instructions like: <em>“Only issue a refund when the customer explicitly requests one and the order meets the refund policy.”</em></p>
<p>You test it, red-team it, and decide the agent’s behavior is acceptable.</p>
<p>Three months later, someone changes the instructions: <em>“Resolve customer complaints proactively. Prioritize customer satisfaction.”</em></p>
<p>The agent’s identity hasn’t changed. It still has the same <code>refund:create</code> permission. But the condition under which it may choose to issue a refund have.</p>
<p>So a review of the existing identity access can answer: <em>“Does this agent still need permission to issue refunds?”</em></p>
<p>But it can’t answer: <em>“Has anything changed that could alter when or how this agent exercises its existing privileges?”</em></p>
<p>If the answer is yes, then your previous security evaluation may no longer be valid.</p>
<p>As you discover that the new model is more aggressive about issuing refunds, you may decide that refunds over #50 require a human approval.</p>
<p>The bottom line is that this necessitates a second layer of governance for persistent agents:</p>
<blockquote>
<p><em>Changes to the model, instructions, tools, or decision context should be treated as security-relevant changes, even when the identity and permissions stay the same.</em></p>
</blockquote>
<p>Practically speaking, this could mean:</p>
<ul>
<li>versioning the agent, not just the application code,</li>
<li>more granular specifications for acceptable agent behaviors for better re-validation,</li>
<li>mechanisms to trigger re-evaluation of security-sensitive agent changes,</li>
<li>deterministic controls around consequential actions</li>
<li>and monitoring behavior, not just authentication - <em>which agent version acted, what tool it used, what resource it changed, etc.</em></li>
</ul>]]></description>
      <pubDate>Fri, 02 Oct 2026 13:22:13 GMT</pubDate>
    </item>
    <item>
      <link>https://aiforthecurious.ai/feed/sql-can-filter-by-meaning</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/sql-can-filter-by-meaning</guid>
      <description><![CDATA[<p>I used to think SQL could only match keywords. This week I learned it can filter by meaning: Quail, an open-source “AI-SQL” engine, lets a query ask an LLM a yes/no question about each row.</p>

<p>Here’s the difference. To find every movie review that talks about the ending, I’d write:</p>
<p><code>WHERE text LIKE '%ending%' OR text LIKE '%climax%'</code></p>
<p>But then I could miss the following review: <em>“I couldn’t believe who the killer was in the final ten minutes.”</em></p>
<p>Quail (QUery Aware Inference Layer) is an open-source engine for “AI-SQL” - queries that can call an LLM inside the <code>WHERE</code> clause. Instead of matching words, you ask the model a question about each row, roughly:</p>
<figure data-rehype-pretty-code-figure><pre tabindex="0" data-language="sql" data-theme="github-light github-dark"><code data-language="sql" data-theme="github-light github-dark" style="display:grid;"><span data-line><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;" data-token-type="comment">-- pseudocode, not real syntax</span></span>
<span data-line><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;" data-token-type="keyword">WHERE</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> ask_model(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;" data-token-type="string">'Does this review discuss the ending?'</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">, review_text)</span></span></code></pre></figure>
<p>The model reads each review and returns true or false.</p>
<p>So as you can see, SQL can filter by meaning now (semantic filtering), not just match strings.</p>
<p>But the catch here is cost. That one query makes one model call per row, so a million reviews means a million LLM calls. Quail is built to make that faster.</p>
<p>On one large multi-join query in their benchmark, a standard vLLM setup took 6.84 hours and Quail took only 29 minutes. Across all 29 benchmark queries, the average speedup was 1.84x. Either way, it’s still a batch job, not a single lookup.</p>]]></description>
      <pubDate>Thu, 01 Oct 2026 20:52:46 GMT</pubDate>
    </item>
    <item>
      <title>Your Team Wants To Build An AI Feature. Now What?</title>
      <link>https://aiforthecurious.ai/feed/ai-proposes-your-system-decides</link>
      <guid isPermaLink="true">https://aiforthecurious.ai/feed/ai-proposes-your-system-decides</guid>
      <description><![CDATA[<p>Your team wants an AI feature, and you’re the one who has to work out how it fits the existing codebase, data and security model. Here’s how I think about it, from context and trust boundaries to the deterministic gates that keep a non-deterministic model in check.</p>
<p>This video walks through the architectural decisions you need to make before writing a single line of agent code, using a real scenario (fulfilling a customer order) to show where the model’s job ends and your architecture begins.</p>
<figure><a href="https://aiforthecurious.ai/feed/ai-proposes-your-system-decides"><img src="https://aiforthecurious.ai/feed/media/ai-proposes-your-system-decides-poster.png" alt="Video poster"/></a></figure><p><a href="https://aiforthecurious.ai/feed/ai-proposes-your-system-decides">Watch the video (8:55)</a></p>
<h2>Transcript</h2>

<h3 id="0000-the-model-is-the-easy-part">[00:00] The model is the easy part<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0000-the-model-is-the-easy-part" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>So, your team is about to build an AI feature, and you were tasked with figuring out how it actually connects to your system, but you don’t even know where to start. The fear of being the person in the room who is expected to have the answer for how this thing fits into the existing codebase, the existing data, and the existing security model overwhelmed you, and you started to doubt yourself. Does that sound familiar to you?</p>
<p>Believe it or not, in this case, the LLM model is the easy part. The hard part is the architecture around it, and here’s why. The model is just an API call. So you send text in, and you get text out. You didn’t have to train it, host it, or maintain it. Someone else spent billions of dollars making it intelligent. You’re consuming that intelligence for pennies per request. But accessing to a capable model only solves one part of the problem.</p>
<p>Now, think about what happens the moment you want that intelligence to do something meaningful in your system. For example, answering a simple question from a customer such as “Can you deliver 500 units by Friday?” The model might interpret the request and propose splitting the shipment between warehouses. That is useful reasoning, but fulfilling the request is a different story. It requires additional context.</p>
<p>The model doesn’t know who your customers are, you have to bring it that context, which may require integrating with your CRM, your ERP system, your document store, and decide how fresh that data needs to be, how much to spend, and in what shapes. That’s a staleness problem, an integration problem, and a data modeling problem. On top of that, the model doesn’t know what it’s allowed to see. If you’re in a multi-tenant system, you probably don’t want the model to answer questions about customer B using customer A data, you need to build a scoping layer.</p>
<h3 id="0205-deterministic-gates-around-a-non-deterministic-model">[02:05] Deterministic gates around a non-deterministic model<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0205-deterministic-gates-around-a-non-deterministic-model" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Now this is nothing new. This is the same trust boundary problem you already deal with in every data query path you have written. Except now the user making the queries is a non-deterministic reasoning engine. The model by itself can’t do anything meaningful. Sure, it can produce text, but for that text to become a reserved inventory line, an updated order, or a triggered workflow. You have to build the action layer, and every action introduces potential failures.</p>
<p>Consider what happens if the model proposes two actions, and the first commits before the second is rejected. For example, what if the reservation succeeds but the notification fails? The model doesn’t know when it’s wrong, it would confidently propose something that could violate a business rule. For example, order greater than $500 requires a human approval. So you need validation gates, authorization checks, and spending limit.</p>
<p>In other words, deterministic boundaries around a non-deterministic component. And where you draw that line is an architectural decision with real trade-offs in both directions. Okay, so where do you actually draw that line? That’s the next question. Your first instinct might be to try to make the AI more controlled by constraining its output, narrowing its options adding more rules to its prompt until it behaves more deterministically.</p>
<p>But that’s the wrong direction. If you wanted deterministic behavior, you would just write a rules engine. The whole point of using AI model is that it can handle ambiguity, interpreting vague requests, weighing options, reasoning through trade-offs that a static rule set can’t express. So here is the core model. Let the AI be open-ended where ambiguity is useful. And only put deterministic gates around the parts where business cannot tolerate improvisations.</p>
<h3 id="0411-ai-proposes-your-system-decides">[04:11] AI proposes, your system decides<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0411-ai-proposes-your-system-decides" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>So let’s go back to our example. A customer asks, “Can you deliver 500 units to our Detroit warehouse by Friday?” This is where open-ended reasoning actually earns its keeps. The AI checks inventory across locations and see Warehouse A has 300 units, Warehouse B has 250 units. Now it can reason. Maybe we can split the shipments across both warehouses. Maybe expedite shipping on one leg. Maybe offer a substitute SKU.</p>
<p>Or maybe ask the customer if partial delivery by Friday with the rest on Monday is acceptable. A rules engine would need every one of those paths hard-coded. The AI can explore all those options, and that’s the beauty of the open-ended reasoning. But here’s the critical part: the AI produced a proposed plan, like reserve 300 from Warehouse A and 200 from Warehouse B, and expedite both. That proposal now hits a series of deterministic gates, the inventory service may have to check.</p>
<p>Are 300 units actually available in Warehouse A right now? It involves real queries against real inventory state to get the definitive yes or no answer, not the maybe. The contract service may need to check: is the customer entitled to expedited shipping under their current agreement? The pricing engine calculates actual shipping costs instead of an estimated cost by the model. And finally, the authorization policy checks: is this agent permitted to commit $7,500 of expedited shipping without human approval?</p>
<p>So as you can see here, each side does what it’s good at. The AI handles interpretations and planning. Your system handles validations, authorization, and executions. So, the core pattern here is: AI proposes and your system decides. Now, what we just walked through here, the context layer, the trust boundaries, the validation gate, the action layer, the industry has a name for it, and they call it AI harness.</p>
<h3 id="0626-the-model-is-interchangeable-the-architecture-isnt">[06:26] The model is interchangeable; the architecture isn’t<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0626-the-model-is-interchangeable-the-architecture-isnt" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>Salesforce recently formalized it as a six-capability architecture. But the underlying principle that I want you to take away is this: first of all, the model is interchangeable. The architecture that connects it to your enterprise is the real product. What this means is that two companies can use the same exact model and get wildly different values from it. The difference isn’t the intelligence in the model, as they’re both calling the same API.</p>
<p>The difference is what each company built around it, how well they connect the model to their customer data, their business rules, their workflows, and their security model. That’s connecting architecture is unique to your organization. It’s built on decades of accumulated domain knowledge, data relationships, and business logic that no model provider ships out of the box. And when a better or cheaper model comes out next month, the company that architected a clean separation between their enterprise context and the model layer just simply swapped the model.</p>
<p>By contrast, the company that entangled everything with one provider’s specific API shape has a migration project on their hands, and that is what makes or breaks an architectural design. The second principle for you to take away is: the skill of designing that architecture are the same architectural thinking that you’re already familiar with, but apply to a new kind of component. The AI model is itself just a component in the system.</p>
<p>Yes, it has some unusual properties, like it’s non-deterministic, it needs context fed to it and it can’t enforce its own permissions. And those properties inflate certain concerns such as trust boundaries get heavier, validation gates become more critical, but the skill of identifying which concerns are inflated and choosing the right mechanism and trade-offs for them, that’s the same skill. So, I hope you find this video useful.</p>
<h3 id="0838-wrapping-up">[08:38] Wrapping up<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#0838-wrapping-up" class="internal internal-link"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
<p>If you’re interested in this topic, AI harness, and would like to dive more into it, like the video and drop a comment below so I know what content serves you well. Thank you and I’ll see you next time.</p>]]></description>
      <pubDate>Tue, 29 Sep 2026 15:13:15 GMT</pubDate>
    </item>
  </channel>
</rss>