Blog

Jev: A Decision Model for the Boring, Important Parts of Your Pipeline

Most of the LLM calls I’ve put in production aren’t really asking the model to write anything. They’re asking it to decide: is this ticket urgent, which bucket does this PR belong in, does this comment need a reply. Jev, the first model from TypeSafe AI, is built for exactly that kind of call. It doesn’t generate text at all. It returns typed values, and after reading through the Jev 1.14 release notes and Claire Vo’s beginner episode on it, I think it’s worth a closer look for anyone building agents or data pipelines.

What Jev actually is

TypeSafe calls Jev a System One Model. The name borrows from the fast, intuitive half of Kahneman’s thinking model, and the idea is that a lot of software needs quick judgment calls, not essays. TypeSafe spent about two years in stealth building it and describes the company as building “machine-native intelligence infrastructure for automation.”

The API shape is simple. You send a state (your data, as text or JSON) and a set of named questions. Each question has a type, and Jev answers all of them in parallel against the same state. According to the Jev docs, there are three main question types:

• choice: pick one label from a list you define. Returns the choice, per-label probabilities, and a confidence.
• score: rate the state on an ordered rubric, lowest to highest. Returns a score, probabilities, and confidence.
• noul: is this statement true? Returns a single number between 0 and 1.

Because each question is evaluated independently, adding more questions to one request barely changes latency. That’s a nice property. You can ask “which department?”, “how frustrated is the customer?” and “is this urgent?” in a single round trip.

Here’s roughly what a request looks like, based on the schema in the quick start guide:

curl https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jev-latest",
    "state": "The app crashes every time I open settings. Third time reporting this!",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {
          "billing": "Payments, invoices, refunds",
          "technical": "Bugs, crashes, performance"
        }
      },
      "frustration": {
        "type": "score",
        "instructions": "How frustrated is the customer?",
        "criteria": ["Calm", "Annoyed", "Very frustrated"]
      },
      "is_urgent": {
        "type": "noul",
        "instructions": "This ticket needs a response today."
      }
    }
  }'

The response comes back keyed by your question IDs, each with its typed answer and probabilities, plus token usage. There’s no parsing of free text, no “please respond in JSON” prompt, and no retry loop for malformed output. There are official SDKs too (pip install typesafe-sdk and npm install @typesafe-ai/sdk) if you’d rather not hand-roll HTTP.

Why it’s interesting: the price

The number that got my attention came from Claire Vo’s How I AI episode on Jev for beginners. Jev costs 4 cents per million input tokens, and there’s no output charge. Since the output is just a handful of numbers and labels, that’s most of the bill.

At that price, classification stops being something you budget for. Vo ran Jev over 1,700 pull requests for 9 cents and used pairwise clustering to see where her engineering time was actually going. That’s a task I’d normally skip because it isn’t worth a few dollars of GPT calls plus the time spent cleaning up the outputs. At 9 cents, you just do it.

What people are building with it

The examples from the episode are a good map of where a decision model fits:

• PR categorization: the 1,700 PRs above, grouped into themes to show where effort went.
• Personal meta-analysis: running Jev over your own local Claude Code and Codex sessions to see patterns in how you work with agents.
• Gmail triage: Jev scores incoming email, and a regular LLM drafts the follow-ups only for the ones that matter.
• Product insights graph: in ChatPRD, 1,100 signals turned into around 200,000 classifications.
• Audience dashboard: a searchable dashboard built from 4,500 YouTube comments for close to nothing.
• Voice-to-color emotion app: a real-time demo pairing Jev with the OpenAI Realtime Voice API.

The pattern across all of them is the same. Jev handles the high-volume scoring, routing and labeling. A general-purpose LLM handles anything that needs words. Vo mentions she doesn’t really use Jev alone anymore, and that matches how I’d use it. It’s a cheap, fast filter in front of an expensive model, not a replacement for one.

If you’ve read my post on designing the control loop behind autonomous agents, this is the “should I keep going?” and “which tool next?” kind of decision. Having those come back as a typed value with a probability attached makes termination conditions and routing a lot easier to reason about than parsing a sentence.

What’s new in Jev 1.14

TypeSafe published the Jev 1.14 release notes on October 14, 2026, framed around two audiences: builders who need production reliability, and dreamers who want to experiment. Their answer is a set of “danger zone” flags. Experimental features sit behind API headers, so you have to opt in explicitly, and TypeSafe is clear that they aren’t ready for production.

Composable images (danger zone)

It turns out Jev 1.13 was already multimodal, it just wasn’t advertised. Now you can add images through a find-and-replace in assets, and reference them in state, instructions or criteria. Turn it on with:

TypeSafe-Danger-Zone-Images: true

TypeSafe’s own advice is worth repeating: every model, Jev included, does worse on multimodal input than on pure text. Break your questions into the simplest pieces you can.

The new box type

This one is only enabled for jev-1.14.0. It returns a single bounding box, and it’s the first output type that isn’t just a smarter if statement. I read it as a hint about where System One models are heading: still typed and constrained, but answering more than “which one?” or “how much?”.

Longer context (danger zone)

Jev can handle more context than the production limit allows. TypeSafe capped it on purpose because reliability drops on long inputs. You can lift the cap with:

TypeSafe-Danger-Zone-Long-Context: true

They still recommend keeping context short, and honestly that’s good advice for any classifier. If the decision depends on 50 pages of input, it’s probably more than one decision.

Model aliases and deprecation

• jev-1.14.0 is available now under the jev-preview alias. Improvements are small and broad, with multilingual support expected to gain the most.
• jev-latest moves to 1.14.0 one week after the announcement.
• jev-1.13.0 stays around for a while. The notice gives 60 days for 1.13.0 and 30 days for 1.14.0, with a formal deprecation policy promised before that kicks in.

If you’re in production, pin a version. Pointing a pipeline at jev-latest and letting a model swap happen under you is a classic way to get a silent shift in your label distribution.

Opt-in data sharing and an internal benchmark

You can now opt in to let your queries help TypeSafe find model weaknesses. They say they still won’t train on your queries. They also shared an internal estimate of developer-relevant intelligence, with a lot of caveats: it compares only open-weight models, the methodology will change, and it’s probably biased toward Jev. Their own advice is to trust your evals over theirs, which I appreciate.

Should you try it?

If you have a pipeline where an LLM is being used as a glorified when expression, yes, probably. Ticket routing, content moderation, lead scoring, log triage, agent routing. Those are the places where typed outputs and near-zero cost add up fast.

I’d start small. Pick one classification step you already run with a general LLM, write the same questions as Jev choice or score types, and compare the outputs on a few hundred real examples. Look at the confidence values too. Low-confidence answers are a natural place to hand off to a bigger model or a human.

And I’d leave the danger zone headers alone in anything that matters, at least for now. TypeSafe is telling you they’re rough, and it’s nice when a vendor says that out loud.

An Introduction to Edge Computing and Edge AI

Edge computing means running computation close to where data is produced, on a phone, a camera, a gateway in a barn, instead of shipping everything to a data center first. Edge AI is the same idea applied to machine learning: the model runs on (or next to) the device, so predictions happen in milliseconds, keep working offline, and don’t send raw data anywhere. This post covers the basics, the small models that make it practical in 2026, a quick look at agriculture, and where the research says things are going.

What edge computing actually is

The cloud model is simple: devices collect data, send it to a big data center, and wait for an answer. That works until it doesn’t. Bandwidth costs money, round trips add latency, and some places (a field, a ship, a mine) just don’t have a reliable connection.

Edge computing pushes part of that work back toward the source. Weisong Shi and colleagues framed it this way in Edge Computing: Vision and Challenges (IEEE IoT Journal, 2016), where the “edge” is any computing resource along the path between the data source and the cloud. Mahadev Satyanarayanan’s The Emergence of Edge Computing (IEEE Computer, 2017) adds the idea of cloudlets: small data centers one network hop away from users.

In practice there are a few layers:

• Device edge: phones, microcontrollers, cameras, drones, cars.
• Near edge: gateways, routers, an NVIDIA Jetson sitting in a shed.
• Far edge / regional: telecom edge servers and small regional data centers.
• Cloud: training, aggregation, anything heavy that isn’t time-sensitive.

You don’t pick one. Most real systems split the work across tiers.

From edge computing to Edge AI

Edge AI is what you get when the workload you’re moving closer to the data is model inference. Zhou et al. called it “the last mile of AI” in Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing (Proceedings of the IEEE, 2019), and the reasons they gave still hold:

• Latency. A crop-spraying drone can’t wait 400 ms for a server to say “that’s a weed”.
• Privacy. Audio, health data and photos never leave the device.
• Cost. No per-token API bill and much less upstream bandwidth.
• Availability. It keeps working with no signal.

The catch is obvious: devices have little memory, little power and slow memory bandwidth. So most of Edge AI is about making models smaller without making them dumb. The classic toolbox includes quantization, pruning and distillation (see Han et al.’s Deep Compression), plus newer LLM-specific tricks like activation-aware quantization (AWQ). Training can also stay distributed: federated learning lets devices improve a shared model without uploading their raw data.

The newest small models worth knowing

A couple of years ago “on-device LLM” mostly meant a demo. Now the open model hubs have a solid lineup sized for phones and single-board computers. A few I’d look at first on Hugging Face:

• Gemma 4 E2B / E4B (Google). The “E” stands for effective parameters (about 2.3B and 4.5B). They take text, image and audio input, have a 128K context, and ship as Apache 2.0. Google publishes ready-to-run builds for its on-device runtime in LiteRT-LM, and reports up to 2.2x faster decoding on mobile GPUs with multi-token prediction.
• SmolLM3 (Hugging Face). A fully open 3B model trained on 11T tokens, with the data and training recipe published. The SmolLM3 write-up is one of the best free explanations of how a small model gets trained.
• Phi-4-mini (Microsoft). Phi-4-mini-instruct is 3.8B, MIT licensed, strong at reasoning for its size. Phi-4-multimodal adds vision and speech.
• Others worth a look: Qwen3.5 (0.8B and 4B), Ministral 3, IBM Granite 4.0 Nano, Liquid AI’s LFM2.5 and Llama 3.2 1B/3B.

Phones also ship their own: Gemini Nano on Android and Apple’s Foundation Models framework on iOS. As an Android developer I find that part interesting, because it changes the question from “can I run a model here?” to “which one is already on the device?”.

If you want to try one on your laptop in a minute, llama.cpp can pull a quantized GGUF straight from Hugging Face:

llama-cli -hf ggml-org/gemma-3-1b-it-GGUF -p "Explain edge computing in one sentence."

One warning from recent research. The August 2026 study Large Models for Small Devices (Das et al.) measured compression on real hardware and found that no technique wins everywhere. Q5_K_M quantization held up well for question answering, while pruning hurt it badly. For image segmentation the ranking flipped. Pruning also broke k-quant block alignment, made files 21 to 49% bigger, and slowed a Raspberry Pi by up to 3.4x. They even found a compressed model with 71% accuracy that was putting 97 of 100 answers in the same class. So benchmark your model on your device, with your data. Paper numbers don’t transfer as well as we’d like.

A quick look at Edge AI in AgTech

Agriculture is probably the best argument for Edge AI. Farms are big, connectivity is bad, margins are thin, and decisions are time-sensitive.

Some places where it’s already useful:

• Plant disease detection from a phone photo, offline. This line of work goes back to datasets like PlantVillage, and today runs on MobileNet or EfficientNet-class models.
• Targeted spraying, where cameras on the boom tell crop from weed in real time and only spray weeds.
• Livestock monitoring with collar sensors and cameras that flag lameness or unusual behavior locally.
• Soil and irrigation sensors on microcontrollers that decide when to water without a server.

The March 2026 review Affordable Precision Agriculture by Samanta and Saha surveyed 2023 to 2026 work and found a few patterns. Most precision-ag systems still depend on the cloud, which shuts out smallholders. Cheap microcontrollers (ESP32, STM32) dominate the hardware. Quantization shows up in about half the papers, while pruning and hardware-aware architecture search are rarely tried. And most papers don’t report RAM, latency or energy, so it’s hard to compare anything. Their summary of the direction is “localized inference with centralized training”, which matches what I’d expect: train in the cloud, run in the field.

I wrote about the human side of this in AI Beyond Tech: farms and the data center race. Edge AI is the technical piece that makes a lot of those ideas affordable.

The future of Edge AI

So where is this going? The academic work points in a few directions, and they mostly agree.

Small models become the default, not the fallback. NVIDIA researchers argue in Small Language Models are the Future of Agentic AI (Belcak et al., 2025) that most agent sub-tasks are narrow and repetitive, so a small model is good enough and much cheaper. Large models get called only when needed. That fits the edge perfectly.

Agents move onto devices. The survey Cognitive Edge Computing looks at how reasoning models and autonomous agents can run on constrained hardware, and the broader Empowering Edge Intelligence survey (later in ACM Computing Surveys) covers the on-device model landscape and its open problems.

The network becomes part of the model. Network Edge Inference for Large Language Models (2026) argues that the design problem is shifting from “fit everything on one device” to co-designing model, memory, compute and communication across device, edge server and cloud. Think split inference, speculative decoding with a small draft model on the phone, and KV caches that live on a nearby server.

Measurement gets serious. Das et al. call for tools that profile prefill and decode separately and recommend a configuration per device. Benchmarks like MLPerf Tiny started this for microcontrollers. LLMs on phones need the same discipline, because memory bandwidth, not FLOPs, is usually what limits you.

My own take: the cloud isn’t going away, it’s just becoming the place where models are trained and where the hard 10% of requests go. The other 90% (classify this leaf, summarize this note, answer this simple question) will happen on hardware you already own. For mobile developers that’s good news. On-device AI is turning into a normal part of the app, like camera or location, and the people who understand memory, battery and latency on real devices are going to be useful for a long time.

References

• Shi et al., Edge Computing: Vision and Challenges, IEEE IoT Journal, 2016
• Satyanarayanan, The Emergence of Edge Computing, IEEE Computer, 2017
• Zhou et al., Edge Intelligence: Paving the Last Mile of AI with Edge Computing, 2019
• Das et al., Large Models for Small Devices, arXiv, 2026
• Samanta and Saha, Affordable Precision Agriculture, arXiv, 2026
• Belcak et al., Small Language Models are the Future of Agentic AI, arXiv, 2025
• Network Edge Inference for Large Language Models, arXiv, 2026
• Hugging Face, SmolLM3: smol, multilingual, long-context reasoner
• Google AI Edge, Gemma 4 on LiteRT-LM

AI Beyond Tech: Slower Data Centers, Smarter Human Uses

When people talk about large language models, they usually mean code, chatbots and office productivity. But some of the most useful LLM work is happening far from tech: on small farms in Kenya and India, on construction sites, and in teams trying to cut carbon. Meanwhile, the race to build bigger models in bigger data centers is harming people and the environment, sometimes the very farmers the technology is supposed to help. Both things are true, and we still get to choose which one wins.

The bottleneck is information

In a lot of agriculture, the bottleneck isn’t tractors or sensors. It’s getting the right information to the right farmer at the right time. A 2019 review in Science by Fabregas, Kremer and Schilbach found that advice sent by mobile phone in sub-Saharan Africa and India raised yields by about 4% and the odds of adopting recommended inputs by 22%. But Cole and Fernando’s randomized study of a voice advisory service for cotton farmers in Gujarat is a useful warning: farmers trusted it and changed their practices, yet yields and profits didn’t clearly improve.

LLMs add flexibility: a farmer can ask in their own language, by voice, with a photo of the leaf. That’s the idea behind FarmerChat from Digital Green (paper), which passed 1 million users across five countries in early 2026. In an independent 60 Decibels study, 7 in 10 Kenyan farmers applied its advice within 30 days.

The engineering lesson matters. GPT-4 passes agronomy certification exams, but Microsoft’s RAG vs Fine-tuning study found the real gains came from grounding models in local data. Digital Green fine-tuned on 25,000+ expert-reviewed Q&A pairs. Quality came from local knowledge and human experts, not a bigger model. A 2026 review still flags its one-way design, and a randomized trial across 600 Kenyan villages is now testing whether it moves yields and income. Good. That’s the evidence we should demand.

AI on the construction site

Construction rarely shows up in AI keynotes, but its footprint is huge. Catherine De Wolf of ETH Zurich points out that about half of the materials we consume go into buildings, which produce roughly a third of global waste. Her team uses AI to catalogue materials from buildings slated for demolition and match them with new projects. At UWE Bristol, ASPEC predicts embodied carbon at the planning stage, when changing materials is still cheap.

LLMs fit here for the same reason they fit on the farm: the work is mostly documents. Reading a 300-page spec to flag high-carbon choices, or turning a safety manual into something a crew will read, doesn’t need a frontier model. It needs accuracy, the right documents, and a low cost per run.

AI for sustainability work

Sustainability teams spend much of their time reading supplier reports and reconciling emissions data. For food companies, most of the footprint sits upstream on farms: fertilizer, land use, irrigation. Matching thousands of supplier sheets against farm data is tedious, text-heavy work, exactly where LLMs shine. They let a small team do what used to need a consultancy.

The race, and who pays for it

While farmers and builders find modest uses for AI, the biggest companies are racing to build ever larger models, and the bill lands on people who never signed up for it.

Energy. The IEA’s Energy and AI report projects data center electricity demand will more than double by 2030, to around 945 TWh, slightly more than Japan uses today. In the US, data processing is expected to use more electricity than aluminum, steel, cement and chemicals manufacturing combined. MIT researchers describe the industry as being on an unsustainable path.

Water. Here the farm and data center stories collide. Agriculture already accounts for around 70% of global freshwater withdrawals, and data centers now compete for the same water, often in dry regions. In Making AI Less “Thirsty”, Shaolei Ren and colleagues estimated that training GPT-3 could have evaporated about 700,000 liters of freshwater, and projected AI could account for 4.2 to 6.6 billion cubic meters of water withdrawal in 2027. Ren also notes that companies are secretive about their energy and water use. A farmer in a drought year can’t be secretive about water. Neither should a hyperscaler.

Air and health. A Caltech and UC Riverside study estimated that air pollution from AI’s power demand could cost the US nearly $20 billion a year in health damage by 2030, linked to around 1,300 premature deaths. The harm falls hardest on low-income communities near power plants and backup generators.

A real example. In South Memphis, xAI powered its Colossus supercomputer with gas turbines that, according to the Southern Environmental Law Center, ran without public notice or oversight. The county approved a permit for 15 turbines in July 2025 after months of opposition, and the NAACP appealed. This neighborhood has lived with industrial pollution for decades. Moving fast meant someone else breathing the exhaust.

Safety. In the Future of Life Institute’s Summer 2026 AI Safety Index, none of the nine companies graded scored above a C+, and the report found that companies have been weakening safety commitments as their models grow more capable.

Let’s be honest about what’s going on. AI companies are big corporations chasing money and power. That’s wild capitalism, and AI isn’t an exception. But I don’t think the technology itself is the problem. The internet gave us open knowledge and a way to stay close to family across the world. It also gave us surveillance, scams and misinformation. Same tech, very different outcomes. I believe AI is the same, and that we can point it at the good things. That’s what I’m working towards. As a tech enthusiast, I want to use technology for good.

What good looks like

The uses that actually help people don’t need any of that excess. Take diabetic retinopathy, a leading cause of blindness that’s easy to miss where eye specialists are scarce. Researchers in Bangalore tested Medios, an AI model that runs offline on the smartphone attached to a retinal camera. No cloud, no internet. It caught 98.8% of referable cases and 100% of vision-threatening ones according to the study in the Indian Journal of Ophthalmology. Or a farmer in Kenya getting a straight answer about fall armyworm in Swahili, on a cheap Android phone, over WhatsApp. An architect reusing steel beams instead of ordering new ones. An analyst finding the 20% of suppliers behind 80% of the emissions. Real wins for food and the planet, and all small, specific tasks.

AI is getting cheaper. So why the endless buildout?

Stanford’s 2025 AI Index found that the cost of querying a GPT-3.5-level model fell from $20 per million tokens in November 2022 to $0.07 by October 2024, a drop of more than 280 times.

And research says we can go further without bigger infrastructure. In Power Hungry Processing, Sasha Luccioni, Yacine Jernite and Emma Strubell found general-purpose generative models are orders of magnitude more expensive than task-specific ones for the same task. A UNESCO and UCL report showed that smaller task-specific models, shorter prompts and quantization could cut energy use by up to 90%. In 2026, UNESCO, France and India launched a global competition to compress models without losing performance. Ren’s group has shown that scheduling workloads during cooler hours reduces water lost to evaporation.

Put that next to the agriculture research and the picture is clear. What made agronomy answers better was local data and expert review, not scale. A farmer’s pest question is short, specific and repeated across thousands of farms. That’s the ideal workload for a small, grounded model, ideally one that runs on a phone with bad connectivity.

So who is the next hyperscale data center really for? Mostly the race to the next frontier model. Not the farmer. That’s why I think we should freeze the data center race:

• Pause new hyperscale AI data centers until companies must publicly report energy, water and air pollution per facility.
• Efficiency first. Default to the smallest model that does the job before asking for a new megawatt.
• Treat water as shared. Where farms already compete for irrigation, weigh a data center’s water use against agriculture and drinking water, in public, before approval.
• Give communities a real say, with a vote before construction and fair compensation for health costs.

Blockchain already showed it can be done

Crypto went through this same fight first. Bitcoin’s proof of work burns energy by design, and the Cambridge Centre for Alternative Finance estimates mining still uses about 138 TWh a year, around 0.5% of global electricity. Under pressure, the industry did shift: the same 2025 survey found 52.4% of miners’ energy now comes from renewables and nuclear, up from 37.6% in 2022, and coal fell from 36.6% to 8.9%. That’s real progress, though it’s a survey of the miners willing to answer, and total consumption kept rising.

Ethereum went further and changed the design itself. In September 2022, the Merge replaced mining with proof of stake, and the Crypto Carbon Ratings Institute estimated it cut the network’s electricity use by more than 99.98%, from about 23 million MWh a year to roughly 2,600. (The report was commissioned by ConsenSys, an Ethereum company, but the order of magnitude isn’t in dispute.)

The lesson for AI is direct: cleaner power helps, but the biggest wins come from changing the architecture. For Ethereum it was proof of stake. For AI, it’s smaller, task-specific models instead of one giant model for everything.

The honest counterargument

The IEA says data center emissions are small relative to the whole energy sector, and could be offset by reductions AI enables. Agriculture is a plausible place for that, if better advice means less wasted fertilizer and water. A freeze in one country might also just push construction somewhere with weaker rules.

AI on the farm isn’t risk-free either. In Nature Machine Intelligence, Asaf Tzachor and colleagues warned of unreliable data, models that optimize yield at the expense of soil and ecosystems, and security problems at scale. They suggest involving anthropologists and ecologists in design, data cooperatives so farmers own their data, and sandbox testing before rollout. Small models don’t fix that on their own.

And cheaper AI doesn’t automatically mean less energy. When something gets cheaper, we use more of it. That’s why I think efficiency alone isn’t enough and some limits are needed. It’s a real tension, and I’d rather name it than hide it.

Wrapping up

LLMs are already helping industries most of tech ignores, and the best examples are modest, specific and cheap. Decades of research on agricultural advice tell us what works: local knowledge, trusted delivery, and measuring whether farmers actually end up better off. None of that needs a bigger data center. The race for ever larger models is a different project, paid for in water, air and health by people who don’t get a say, including the farmers who need that water most. We have the research and the cheap models. So here’s my question: shouldn’t we stop chasing bigger models and focus on making LLMs use less energy, then put them to work where people actually live, in health, agriculture, education and the places most of tech still ignores?

Loop Engineering in Practice: What No Middleman Taught Me About Verifying AI-Written Code

GitLab’s 2026 AI Accountability Report found that 78% of developers say they’re writing and committing code faster with AI tools. It also found that overall software delivery hasn’t sped up to match. The gap is explained by one number: 85% of respondents say AI has shifted the bottleneck from writing code to reviewing and validating it. Everyone got a faster typist. Almost nobody got a faster team.

That’s the exact problem I’ve been building against with No Middleman, a Claude Code plugin for React Native that lets an agent loop on a feature or bug fix unattended, but only ever hands you a pull request that’s already been proven against a real emulator, not just asserted as done. I wrote about the general shape of agent control loops in a previous post. This one is about the part of that loop that actually matters once you let the agent run without you watching: verification.

Trusting the diff doesn’t scale

The default way most of us review AI-generated code right now is to read the diff and decide if it looks right. That works when the change is small and you already understand the surrounding code. It stops working the moment the agent is iterating on its own, because “looks right” is a judgment about static text, and the failure modes that matter, a race condition, a mock that silently changed behavior, a test that got loosened instead of the code getting fixed, don’t show up in a diff. They show up when the thing runs.

GitLab’s finding lines up with what you’d expect from that: reviewing and validating AI output is slower and harder than writing the prompt that generated it, so the bottleneck moved downstream instead of disappearing. Reading harder isn’t the fix. Verifying differently is.

Assess, act, verify

No Middleman’s loop is a typed LoopSpec run on XState v5: a trigger, an intake (a goal plus the file globs it’s allowed to touch), a composite verification chain ordered cheap to expensive, and a stopping rule with a hard budget. Nothing about “done” is left to the model’s judgment.

nm verify --e2e

That single command is the thing the loop iterates against. It runs typecheck, lint, unit tests, and a build, in that order, fail-fast, before it ever touches an emulator. Only once those pass does it get to the expensive check. Cheap gates first means a broken import gets caught in seconds, not after a five-minute Detox run.

Detox as the oracle, not a formality

The headline gate is a Detox end-to-end flow, and the workflow deliberately puts a human at the point where the test is defined, not where it’s checked. You write the goal, a spec-author agent drafts the acceptance flow, and you approve it before anything starts. That red test is the contract. The agent never gets to negotiate what “done” means after the fact, because the definition was fixed before it wrote a line of code.

The dev client compiles once, cached by a hash of the native inputs, and every iteration after that just reloads JS. That’s what makes a real Detox run cheap enough to sit inside a loop instead of being a once-a-day CI job: the expensive part only happens when something native actually changed.

The checker can’t be the maker

The part I’d guess most people skip is the one that matters most: green isn’t the finish line. Once the suite passes, a separate Opus agent, one that never touched the implementation, reviews the diff specifically hunting for the ways a model games its own tests: weakened assertions, mocks that quietly make the test meaningless, timing masked instead of fixed, scope that crept outside what the goal asked for. It’s an adversarial checker by design, not an optimistic second opinion.

That split exists because letting an agent grade its own homework is exactly how you get a green checkmark on code that doesn’t actually work. If the same model that wrote the fix also decides whether the fix is good, its incentive is to reach green by whatever path is shortest, and loosening a test is a shorter path than fixing the underlying bug. Separating maker from checker removes that incentive.

Invariants, not intentions

The loop won’t even start if a spec violates one of eight hard rules: require-verifiable-stop, bounded-retries, maker-neq-checker, tests-only-strengthen, external-state-only, read-only-by-default, human-approves-acceptance, pr-not-merge. tests-only-strengthen is the one doing the most quiet work: a test can get stricter as the loop iterates, never looser. That closes off the single easiest way an unsupervised agent could cheat its way to a passing run.

pr-not-merge closes the other one. The loop’s output is always a draft pull request, whether the checker accepted the change or the iteration budget ran out first. Nothing merges itself. A hard cap on iterations, tokens, and wall-clock time, plus a reserve the loop never spends on iteration, means it always has enough left to write a clean handoff instead of dying mid-edit. And if something looks wrong while it’s running, touch .nm/KILL halts it between iterations and leaves a clean worktree behind.

The two ends you still own

The name is literal: the loop removes you from its interior, the part where you’d otherwise sit watching each edit-reload-verify cycle and nudging it back on track. It doesn’t remove you from the two ends. You write the acceptance test that defines what “done” means, and you review the PR that comes out. Autonomous interior, human bookends. That’s a different claim than “the agent tested itself,” and it’s the reason I trust the output more than I trust a diff someone asks me to skim.

What this means for reviewing AI code generally

The GitLab numbers describe a team where AI made the input to review faster without making review itself faster, which is a bottleneck relocation, not a productivity gain. The fix isn’t reading diffs more carefully. It’s building a verifier that runs the code and grades it against a test you wrote and a checker that has no stake in the outcome, so that by the time a human opens the PR, the question isn’t “does this look right,” it’s “do I approve of the test that already proved it.”

Loop Engineering: Designing the Control Loop Behind Autonomous Agents

Most of the interesting failures in an agentic system are not in the prompt. They are in the loop. Once an agent is planning, calling tools, checking its own output, and deciding whether to go again, the thing you are actually building is a control loop with an LLM sitting inside it. Loop engineering is the name that has stuck for the discipline of designing that structure on purpose, and it is worth understanding as its own layer, separate from prompt design.

Where the term comes from

The term got its clearest articulation in Addy Osmani’s Loop Engineering essay, which argues that the skill that actually separates reliable agents from flaky demos is not prompting, it’s the surrounding loop: how the agent decides what to do next, how it recovers from a bad step, and how it knows when to stop. That framing is why the term has since collected its own body of references rather than staying folded into general prompt engineering advice. The Loop Engineering Sources page and the Awesome Loop Engineering list on Hugging Face are both attempts to keep that growing literature in one place, and the Agentic AI Knowledge Base folds it into its broader documentation on agent harnesses, which is a fitting place for it: a loop is infrastructure, not a prompt technique.

The anatomy of an agent loop

Strip away the framework abstractions and most agent loops are doing the same four things on repeat:

1. Observe — read the current state: user input, tool results, prior steps.
2. Plan — decide the next action, which might be a tool call or a final answer.
3. Act — execute the action.
4. Check — decide whether to loop again, stop, or escalate.

The fourth step is the one frameworks tend to hide, and it is the one worth the most attention. If the model decides for itself when it is “done,” you need an explicit answer for what happens when it decides wrong, either stopping too early or never stopping at all.

Termination is the whole game

A model that is uncertain will often choose to gather more information rather than commit to an answer, and “gather more information” is an action the loop is perfectly happy to repeat indefinitely. So termination cannot be left to the model alone. In practice, a few overlapping guards catch almost every runaway loop:

MAX_ITERATIONS = 12
MAX_TOOL_CALLS_PER_TASK = 20
MAX_WALL_CLOCK_SECONDS = 300

while not done and iterations < MAX_ITERATIONS:
    action = model.plan(state)
    if is_repeat_of_last_n(action, history, n=3):
        escalate("loop appears stuck, repeating the same action")
        break
    state = execute(action)
    iterations += 1

None of these guards are clever, and that’s the point. A hard iteration cap, a repeated-action detector, and a wall clock timeout are cheap to reason about compared to trying to make the model smarter about knowing when it’s finished.

Stop hand-holding, start designing the loop

There’s a habit that shows up in a lot of early agent projects: a human sits next to the loop, watching each step, nudging it back on track whenever it drifts. That works for a demo. It does not scale, and it quietly hides the fact that the loop itself has no real recovery strategy. The paper Stop Hand-Holding Your Coding Agent pushes on exactly this: the fix for an agent that needs constant supervision is not more supervision, it’s giving the loop the guardrails, checkpoints, and escalation paths that let it run unattended in the first place. If your agent only works when someone is watching it, the loop design is the thing to fix, not the babysitting routine.

Loops vs. deterministic graphs

A freeform while loop where the model decides every next step is flexible, but flexibility and reliability trade off against each other as the task gets more complex. The paper From Agent Loops to Deterministic Graphs makes the case for pulling structure out of the model’s head and into an explicit graph: known transitions become edges, and the model is only asked to make a decision at the points where a decision genuinely needs judgment. This mirrors something that’s easy to miss when you’re deep in an unstructured loop: not every step needs an LLM call. The steps that are deterministic should be written as deterministic code, and the loop should reserve the model for the parts that actually require reasoning.

Robustness as a property of the loop, not a single run

A loop that works once is not the same as a loop that works reliably over weeks of unattended operation. The paper Engineering Robustness into Personal Agents with the AI Workflow Store looks at this from the angle of personal agents that run continuously in the background, and argues for treating recurring workflows as reusable, versioned artifacts rather than re-deriving the plan from scratch on every run. That’s a useful reframe: a lot of what looks like “agent reliability” is really “workflow reliability,” and workflows that have been run, checked, and refined once are a better foundation than a loop that starts from a blank plan every time.

Context is a loop problem

The other place loops fail quietly is context. Every iteration appends to the state the next iteration reads, so a loop that runs long enough eventually chokes on its own history. Two things matter here:

• Compaction, not just truncation. Summarizing the last several tool results into a paragraph keeps the signal and drops the noise. Dropping the oldest turns blindly tends to drop the reason the agent started the task in the first place.
• Cache-aware batching. If your provider caches prompt prefixes, an iteration that reads back its own history within a few minutes is cheap. One that reads it back an hour later pays full price, which changes how you should schedule retries or background steps in a long-running loop.

A lot of what gets labeled “agent reliability” is really “context management” wearing a different name. A loop that’s correct on iteration two, verbose by iteration six, and hallucinating a tool result on iteration ten usually isn’t a reasoning failure. It’s a context window that scrolled past the thing that mattered.

Fixed intervals vs. self-pacing

There’s a design choice that shows up in almost every long-running agent: does the loop wake up on a fixed schedule, or does it decide its own pace? Fixed intervals are simple and predictable, which is right when you’re polling something external that changes on its own clock, like a CI run or a data feed. Self-pacing fits better when the loop is waiting on its own work: it can reason about how long the next step will plausibly take and schedule accordingly, instead of burning cycles checking on something that has no reason to have changed yet. Defaulting to a short fixed interval for everything, on the theory that checking more often is safer, usually just makes the loop more expensive without making the task finish any faster.

Failure handling belongs in the loop, not the prompt

The instinct when a tool call fails is to describe the error to the model and ask it to recover. That works for the failures you anticipated. It does not work for the ones you didn’t, which is most of them once a system reaches production. What holds up better is treating failure classes as loop-level branches instead of prompt-level instructions:

try:
    result = execute(action)
except TransientError:
    retry_with_backoff(action, max_retries=2)
except SchemaError:
    escalate_to_human(state, reason="tool contract violated")
except Exception:
    escalate_to_human(state, reason="unhandled failure")

The model gets to be clever about the task. The loop stays boring and predictable about failure. That split is deliberate, and it’s why most of the interesting bugs in a mature agent trace back to a loop that had no opinion about failure until it was already in one.

The checklist

1. Cap iterations, tool calls, and wall clock time explicitly. Don’t trust the model to know when it’s done.
2. Detect repeated actions. A loop that calls the same tool with the same arguments twice in a row is stuck, not thorough.
3. Push deterministic steps out of the model and into code. Reserve the LLM call for the points that actually require judgment.
4. Compact context instead of truncating it. Summarize what happened, don’t just chop off the beginning.
5. Match the wake-up interval to what you’re actually waiting for. External clock, fixed interval. Your own work, self-paced.
6. Push failure handling into the loop, not the prompt. Classify errors, decide programmatically, escalate what you can’t classify.
7. Treat recurring workflows as reusable artifacts, not something the agent re-derives from scratch every run.

None of this is exotic. It’s closer to the discipline you’d apply to a retry queue or a background job processor than anything specific to LLMs. That’s probably why it gets skipped: it doesn’t feel like AI work, so it doesn’t get the attention the prompt does. It should.