For AI agents: what is on every page → thkdigital.com/sitemap.xml · Guide → thkdigital.com/agent.md · Rules → thkdigital.com/robots.txt

THK Digital
FIG P.02 · Working paperJune 2026

Everyone is Hammering Screws.

This paper makes a single load-bearing claim: where a computation physically runs fixes what it costs and how long it takes, and no amount of clever software above that layer can unfix it. Infrastructure-first architecture is the discipline of routing each computation to the silicon that handles it cheapest and fastest, and reserving the most expensive silicon in the building for the one job that actually requires it. The industry's bill is not the price of intelligence. It is the price of swinging the hammer at every job.


Section 1

Thesis

Thesis 1 made an architectural argument: agentic AI in production succeeds or fails on the infrastructure surrounding the model, not the capability of the model itself. That argument is now being made back to me, in different words, by the most influential capital in the market. What it leaves unsaid is why the architecture works, and the answer to that question does not live in the architecture. It lives a layer below it, in the silicon. The standard reading treats infrastructure-first design as a matter of engineering taste, a cleaner way to build the same thing. It is not a matter of taste. It is a matter of physics and arithmetic, and once the arithmetic is on the page the architecture stops looking like a preference and starts looking like the only configuration that survives contact with a budget.

This paper makes a single load-bearing claim: where a computation physically runs fixes what it costs and how long it takes, and no amount of clever software above that layer can unfix it. A GPU and a CPU do not bill the same way, do not move data the same way, and do not finish the same instruction in the same number of nanoseconds. Infrastructure-first architecture is the discipline of routing each computation to the silicon that handles it cheapest and fastest, and reserving the most expensive silicon in the building for the one job that actually requires it. Get the routing right and the system runs on commodity hardware with a frontier model touched only at the edges. Get it wrong and every trivial decision in the workflow inherits the pricing curve and the latency floor of the most expensive chip you own.

I did not arrive at this from the silicon literature. I arrived at it from a constraint. The production scheduling agent I built and shipped across seven months, alone, on a fixed six-hundred-pound monthly envelope, reached its final architecture only when the deterministic tooling took over the work the model had been doing inside its prompt: the travel-time calculations, the slot-window arithmetic, the constraint checks. The model was left with bounded classification, and the inference workload dropped far enough that a mid-tier model could run the job in seconds rather than minutes. The lesson did not register as a silicon lesson at the time. It registered as a cost lesson and a latency lesson. Only later did it become obvious that the two are the same lesson, and that both were decided the moment a calculation moved off the GPU and onto the CPU where it belonged.

Consider an illustrative case, kept deliberately generic so the shape is the point and not the domain. A company orders roughly twenty thousand parts a week and needs to route each order to the right supplier against stock, lead time, and cost. There are two ways to build it. The first wraps every step in a model call: the model reads the order, the model checks availability, the model compares lead times, the model picks the supplier. Every one of those steps is arithmetic and lookup, and every one of them now runs on a GPU and bills like one. The second builds a deterministic spine on the CPU that does the reading, the checking, and the comparing in microseconds, and calls a small bounded model exactly once, for the single decision that genuinely needs judgement. Same workflow. Same output. The two systems differ by an order of magnitude in cost and in latency, and the difference was set entirely by where each step was allowed to run.

The rest of this paper builds that argument from the substrate up. Section 2 takes the compute substrate directly: what a GPU-hour buys against a CPU-hour, and what it costs to route CPU-shaped work through GPU silicon. Section 3 takes the retrieval substrate: where query latency comes from, and why the data layer earns its place on speed and not only on correctness. Section 4 widens the lens to the industry, where the same everything-is-a-hammer pattern is being financed at the scale of a capital war, and asks what that hammer actually costs the people swinging it. Section 5 argues that efficiency at this layer is a choice rather than a default, and that the choice is behavioural before it is technical. Section 6 works a full example scenario end to end. Section 7 takes up what comes next: self-hosting the bounded model on hardware a company owns, the data residency that ownership secures and the near-zero attack surface the bounded context gives, and what the whole argument means for the companies being built now.

Section 2

The Compute Substrate

Every cost in an AI system resolves to a chip running for a length of time. This section is about the chip. Section 3 is about the time. The chip comes first, because its price is the one part of the system the software above it cannot argue its way out of. No amount of clever orchestration changes what an hour of the silicon costs. A production system runs on a short menu of it, and the prices on that menu are not close to one another.

The CPU core-hour, cheap and doing real work. Commodity processor time is astonishingly cheap. A dedicated-vCPU server from a European host like Hetzner starts in the low tens of euros a month; the same capacity from a US hyperscaler runs closer to five cents per vCPU-hour (vendor pricing, June 2026). What matters more than the price is that the cheap chip is not a toy. A single modern server CPU with wide vector instructions sustains hundreds of high-dimensional similarity queries a second at ninety-nine per cent recall (an independent pgvector benchmark from 2024), and the current server silicon is marketed by its own manufacturer for running language models at the thirteen-billion-parameter tier and below without a GPU at all (vendor positioning). This is the clerk of the system. A fixed, modest wage, and ordinary work done reliably all day. Most of what a production system does is ordinary work.

The GPU-hour, dear and scarce. A flagship data-centre accelerator does not rent in cents. A current-generation card runs somewhere between two and nine dollars an hour depending on the provider, a spread of roughly thirteen times across the market for identical silicon (aggregated from rental trackers, moves weekly). Either way it is several times the price of a CPU-hour, and for the steady-state work that fills most of a system’s day that gap only compounds. It is dear for a real reason, and the honest thing is to say so plainly. The GPU does massively parallel floating-point arithmetic that a CPU cannot come close to matching, and for training a model or serving a large one at volume there is no substitute. Conceding that is not a weakness in the argument. It sharpens it. The GPU earns its rate on the work that is genuinely its own. The question the rest of this paper presses is how much of a production system’s work is that work, and how much of it is sent to the specialist out of habit. Scarcity makes the rate worse. New data-centre cards have run to lead times of a year or more in some channels, with the high-bandwidth memory and advanced packaging they depend on allocated well into 2027 (analyst reporting, channel-dependent and contested). The specialist is not only expensive by the hour. The specialist is booked.

The TPU, real but vendor-measured. A third option exists at the top of the market. Google’s latest tensor-processing chip reached general availability at the end of 2025 with high-bandwidth memory and peak throughput in the same class as the fastest GPU (trade-press and vendor figures), and Anthropic has announced access to as many as a million of them. Two cautions keep it in proportion. It is a hyperscaler-scale instrument reachable through a single cloud, not a chip a smaller operator buys outright and racks in its own building. And its reputation for efficiency runs ahead of the evidence. The widely repeated claim that it is several times more energy-efficient than a GPU traces to no primary source; the one peer-reviewed comparison available measures an older generation against an older GPU and implies a far more modest advantage (that figure belongs to its authors, not to this paper, and the original is the reference). The TPU is a scale option whose efficiency claims need independent validation before anyone leans on them. It is not a rate a typical system plans around.

One fact sits beneath all three prices and pushes the cheap and the dear further apart. Since the autumn of 2025 the memory market has been in a structural shortage. The makers redirected manufacturing capacity toward the high-bandwidth memory that AI accelerators need, and the rest of the market has been paying for it since. A major European host has warned of double-digit rises on new cloud capacity and far larger rises on memory line-items specifically, and spot prices for ordinary server memory have climbed by triple digits inside a year (vendor and analyst figures). The clearest tell that this is a shortage and not an ordinary cycle is that consumer graphics cards have grown more expensive in the middle of a generational upgrade, when prices normally fall: a card with a two-thousand-dollar list price has been selling for half as much again or more (street pricing, volatile and regional). The same scarcity that lifts the GPU lifts the floor under everything with memory in it. The gap between cheap and dear silicon is not closing on its own. The supply chain is holding it open.

None of this is a bill yet. A rate is a price per hour, and nobody is charged for an hour in the abstract; the charge is for the hours the work actually consumes. A cheap clerk sitting idle still draws the wage. An expensive specialist set to do an hour of a clerk’s work still charges the specialist’s fee for the hour. The rate fixes what an hour is worth. What fixes how many hours the work takes is the other half of the equation, and it is the half this section has left deliberately untouched: time. Section 3 takes it up.

Section 3

The Retrieval Substrate

Section 2 priced the chip. A CPU core-hour is cheap and does real work. A GPU-hour is dear, and scarce. That is a rate, and a rate on its own is not a bill. Nobody pays for a chip-hour in the abstract; they pay for the chip-hours their work actually consumes, and that number is set by the one thing Section 2 deliberately left alone: time. How long the work occupies the chip that was rented. Multiply the rate by the time, and the bill follows.

This section is about time, and time has one advantage over cost: it can be measured directly. Speed is distance over time, the oldest identity in physics. Run it on a processor and the distance a query travels is the volume of data it has to move, while the speed is fixed by the silicon Section 2 already priced. Rearrange it and latency is what remains: for a chip held at a constant rate, the time a query takes is a direct readout of how much data it touched. The quantity that matters is invisible. Latency is the dial that shows it. That is what makes latency the measurable for everything that follows. And on rented compute, latency carries a second property the rest of this section turns on: time is the unit the bill is written in.

Time saved is data not moved. What follows is the engineering of moving less.

So picture the clerk from Section 2, the cheap one, sent to fetch what the next step of the work needs. The clerk moves at a fixed pace. The only things that decide the bill are how far the clerk has to walk and how much is carried back. Shorten both and the latency falls, and on a rented processor a lower latency is the entire return on the hour being paid for. Everything that follows is a way to shorten the walk.

The index as a filing system. Without one, finding a record means reading every record: a sequential scan over the whole table, its cost rising in step with the data. A B-tree turns that walk into a handful of steps, because the tree is shallow and its cost grows with the logarithm of the table’s size rather than its length. The effect is not marginal. One documented case took a query on a roughly five-hundred-million-row table from about twenty seconds on a sequential scan to about two milliseconds once a B-tree index was added (an independent case study). Going from five million rows to five hundred million adds a level or two to the tree and almost nothing to the lookup. When every column a query needs is already held in the index, the clerk never walks to the shelves at all.

Semantic search as a graph, not a sweep. Matching by meaning rather than by exact text is the one retrieval task that looks as though it must be expensive, because a query vector has to be compared against millions of others. It is not, for the same reason the B-tree is not. The HNSW index (Malkov and Yashunin; the original paper is the reference for its parameters, not this one) gives logarithmic search: a query takes a few hops through sparse upper layers, then a short search of a small neighbourhood at the bottom, visiting hundreds of candidates rather than the whole collection. Measured single-query latency on pgvector sits around two milliseconds at a hundred thousand vectors and around eleven at two million (a single independent benchmark on synthetic vectors). The scaling is the point: a twentyfold increase in the collection raised that latency roughly fivefold, not twentyfold.

The filter as a near-free test. Real queries are seldom “find the nearest neighbours” alone. They are “find the nearest neighbours that are also in stock, in this region, under this price.” The constraint can be applied before the graph search, after it, or during it, and the method now shipping in production applies it during traversal (pgvector’s iterative scan since late 2024, and ACORN-style integration elsewhere), at a measured cost of roughly two to four times a plain search. The reason that overhead exists is the part worth understanding. The per-node test is almost nothing, a sub-microsecond check against a bitset. The cost is that a restrictive filter forces the search to visit more nodes before it finds enough that pass. The price of a query is set less by its arithmetic than by how much of the collection the question forces the clerk to touch.

SIMD, where the arithmetic all but vanishes. Even the comparison at the centre of a vector search, the distance between two high-dimensional vectors, is cheap on a modern CPU. One AVX-512 instruction works on sixteen 32-bit floats at once, so a distance over fifteen hundred dimensions resolves in a few microseconds, and pgvector has dispatched to these instructions since version 0.7. The arithmetic was never the expensive part. The data movement around it is, which is why every mechanism above is about reading less rather than computing faster.

Put the pieces together and a tuned retrieval path on commodity CPU stays in the microsecond-to-low-millisecond band from a hundred thousand vectors to several million. Set that beside the thing it feeds. A small model’s time to first token runs in the low hundreds of milliseconds, and a full response in the high hundreds to a couple of thousand (Mistral 7B at roughly 130 milliseconds to first token and 700 to finish, on the benchmark in the research). Retrieval runs one to three orders of magnitude below the model it serves. It is rarely the bottleneck. The slow step in a request is the probabilistic one, not the fetch.

Two things follow, and they are the reason the section exists. The first is that retrieval almost never needs a GPU. On the cleanest primary comparison available, CPU graph search returns a single query in well under a millisecond on five million high-dimension vectors, and the GPU improves that to a fraction of a millisecond (Meta and NVIDIA). The GPU’s real advantages are in building the index, up to an order of magnitude faster, and in serving thousands of queries at once, and neither of those touches the latency of one online request. The crossover only arrives past roughly ten million vectors on a node, or thousands of concurrent queries a second, or a workload that rebuilds its index constantly. The operator in Section 6 hits none of them. The second is that the fetch becomes a bottleneck only when it is allowed to. In one measured example a careless filter turned a query that should have returned in under a millisecond into one that took over four seconds: same data, same hardware, one bad decision about how much to move.

A physical consequence rides behind the latency, and it has to stand on its own legs rather than be bolted onto the equation. A constant-rate processor does not run hotter when it does more work; it runs for longer. But moving fewer bytes does mean fewer transistors switching and less data shuttled across the chip, and data movement is the dominant energy cost in modern compute. Less data moved is less energy drawn, and the energy that is drawn leaves as heat that costs again to remove. Compute energy and cooling energy both fall. There is no clean published figure for the size of that effect, and the argument does not need one; it follows from the physics. For an operator who owns the hardware rather than renting it, lighter and cooler running is a secondary benefit, though a softer one than the energy claim and not to be oversold.

For everyone renting the processor the accounting is simpler and harder to argue with. The meter runs on time, the time is set by the data moved, and the data moved is a design decision. A retrieval layer built to move less is a retrieval layer that costs less, running on hardware that already costs a fraction of the alternative. Which raises the question the next section has to answer. If the substrate is this cheap and the work this light, where is the industry’s money actually going?

Section 4

The Cost of the Hammer

Section 3 left us here: AI is neither the cheapest way to do this work nor the most efficient. So where did the hype come from? Why is it the only thing anyone is talking about, and how are companies raising this much money behind it? Is any of it earning a return, or are people just following the noise and reaching for the same tool no matter the job?

The old line about the hammer is that if it is the only tool you own, everything starts to look like a nail. What is happening across the AI market is worse. The better tool is sitting in the drawer, and companies reach past it for the hammer anyway, because the hammer works and you can watch it work, and deciding which jobs actually need it is harder than swinging.

And lately they have been handed a reason not to look in the drawer at all. The industry has spent the past year teaching the market a single idea: that intelligence is a commodity now, cheap and abundant and getting more so by the month. The largest companies in the field say it plainly and build as though they believe it. When a thing is sold to you as limitless and nearly free, you stop asking whether the task in front of you ever needed it. That is the permission. Laziness explains why one person reaches for the hammer; the commodity story explains why an entire market stopped feeling the need to check.

Here is the claim, and the rest of this section is the case for it. The bill the industry is now staring at is not the price of intelligence. A large part of it is the price of laziness, and the chain from one to the other is short. Swinging the hammer at every task burns compute the task never needed. That compute needs more chips and more data centres to hold them. The data centres need capital, an amount of capital with no precedent, and the people writing the cheques have started to ask what they are getting back. The evidence proves two of those links cleanly: the capital is real, and the return is not arriving. The link in between, that a meaningful slice of this demand is self-inflicted, is my argument and not a cited fact. The case is what follows.

Start with the scale, because it is hard to take seriously until the numbers sit next to each other. In 2024 the four largest hyperscalers spent a little over $200 billion between them on this build. Morgan Stanley puts the combined figure for 2026 at around $740 billion, up roughly 69 per cent on the year before. That is not steady growth. It is a near-quadrupling in two years, and a third consecutive year of spending rising by more than 60 per cent. The market has asked the industry one question, which is how to meet demand, and the industry has given two answers. The first is to build more capacity, more chips and buildings and power, until the commodity is cheaper to produce. The second is to make the model sharper, until there is more the commodity can plausibly be sold to do. Cheaper, and sharper. Both forge a better hammer. Neither answer asks the question that comes before both of them: whether the work needed a hammer at all.

The tell that some of this demand is manufactured rather than real is that the companies best placed to judge are pulling back. The Verge and Fortune reported in May 2026 that Microsoft was cancelling most of its direct Claude Code licences inside the division responsible for Windows, Microsoft 365, Outlook, Teams and Surface, moving its engineers onto a different tool by the end of June. This was not a retreat from AI. The Claude models still run inside the tool Microsoft moved to, and the wider arrangement between the two companies was left untouched. What Microsoft cut was a specific kind of spend. The pricing was token-based; engineers ran the agent for hours on hard problems; the tokens piled up faster than the work could justify them. The honest way to put it is that the spend became impossible to defend at the scale the engineers were using it, and a company that can afford almost anything decided this particular thing was not worth the bill.

Microsoft is not the only one, and the corroboration is sharper. Uber’s chief technology officer said in April that the company had burned its entire planned 2026 budget for AI coding tools in four months. Use of the same tool had climbed from a third of his five-thousand-strong engineering organisation to more than four-fifths in a matter of months, with individual engineers spending between $500 and $2,000 a month on tokens. Then there is the cleanest single illustration in the pile. Axios reported an AI consultant’s account of a client that spent $500 million in a single month on Claude, after failing to set any usage cap on its employees’ licences. The figure is single-sourced and the client is anonymous, so I will not present it as proven. What survives is the image from the same reporting: employees had been using a frontier model for trivial things, including checking the weather. Someone, somewhere, was spending frontier-model money to find out whether it would rain. That is the hammer in one line.

The bills keep climbing even as the technology gets cheaper, which sounds contradictory until you see why. Epoch AI’s measurements put the fall in the price of a given slice of model output at anywhere from single digits to several hundredfold a year, depending on the task. Enterprise AI bills have risen anyway. CloudZero put average monthly AI spend at $63,000 in 2024 and $85,500 in 2025, a rise of more than a third in a single year. The reason is the one this paper keeps returning to. Cheaper tokens do not make a cheaper system when you point the expensive engine at everything and let each task swallow thousands of tokens it never needed. Gartner has said it plainly: the falling token price will not become cheaper enterprise AI, because the new workloads consume far more tokens and providers will not pass the savings through in full. An analysis of more than two billion enterprise API calls found that routing every call to a frontier model cost around $18 per million tokens, against around $2 with tiered routing, a gap of 87 per cent from one architectural decision. The companies paying the $18 are not buying more intelligence. They are buying the same answer at nearly eight times the price.

The cost does not stop at the invoice. More compute needs more power, and the bill for that power has begun to land on people who never signed up for it. Goldman Sachs found that electricity prices in the United States rose just under 7 per cent in 2025, more than double headline inflation, with data centres behind around 40 per cent of the growth in demand. The market monitor for the largest US regional grid attributed some $23.1 billion of capacity cost to data-centre load and called it a wealth transfer. In Manassas, Virginia, a long-standing resident opened a January electricity bill of $281, up from about $100 the month before. The build grew so large that feeding it became a political problem. In March 2026, after previewing it in the State of the Union, President Trump had seven of the largest AI companies sign what was called the Ratepayer Protection Pledge, a commitment to cover the cost of the new power their data centres need so it does not fall on household bills. The pledge was signed. It is also voluntary, carries no audit and no enforcement, and independent lawyers and analysts have called it toothless. A promise with no mechanism is worth pointing at for one reason: the bet has grown so large that its costs are now being negotiated inside the White House.

I want to be precise about what the power story proves and what it does not. Aggregate data-centre demand is straining the grid, and households are paying for part of it. I am not claiming that one company checking the weather on a frontier model moved anyone’s electricity bill; the wasteful slice is not visible as a line item. The honest version is narrower and still damning. Inside an aggregate cost large enough to raise prices on millions of homes, the portion spent swinging the hammer at work that never needed it is pure deadweight, paid for by people who were never in the room.

So this is where the hype has carried us. A bill of $740 billion a year and climbing, with the return still missing. The companies best able to afford it are quietly cutting the spend they cannot justify. The cost of the part nobody is justifying is pushed outward, onto customers through token prices and onto households through power bills.

There is a last irony here, and it carries the whole argument in one word. Jensen Huang, whose company sells more of the commodity than anyone, put it plainly at NVIDIA’s developer conference in March 2026: “Tokens are the new commodity. And like all commodities, once it reaches an inflection, once it becomes mature or becomes maturing, it will segment into different parts.” He is right, and the word he chose convicts the practice. A commodity is the one thing you are meant to meter, to grade, to match to the job; segmentation is exactly the discipline of sending the cheap grade to the cheap task. Naming it a commodity should have been the beginning of that discipline, not the end of it. Instead the word became the permission. The answer to a hammer that costs too much has been a cheaper hammer and a sharper one, never the tool that was in the drawer the whole time. More capacity is not a fix. It is more of the cause. The question that matters is not how much more to build. It is how little of this ever needed building.

Section 5

Efficiency Is a Choice

So what is the actual solution?

Complaining that AI is expensive is the easy part, and the internet is full of people doing it. If a long account of why the bills are too high were all I had, with no way out at the end of it, I would be one more voice in that pile. I’m not. I took you down to the silicon and back up through the capex so that when I state the fix you would already know why it holds.

The fix is not “stop using AI.” Future automation will need it, and the work it does well is work nothing deterministic can touch. The fix is narrower and harder than that. It is tightening when you reach for it.

Go back to the hammer. The problem in Section 4 was never that companies own a hammer. It is that they swing it at everything. A hammer is the right tool for a nail. It is the wrong tool for a screw, it is the wrong tool for a bolt, and no amount of swinging turns it into the right one. The discipline the whole market has skipped is the simplest one there is: knowing a nail when you see it.

A nail is genuine judgement. It is the work where the input is ambiguous, where no fixed rule returns a reliable answer, where you need something that can weigh a case it has never seen before. That is what a language model is for, and it is very good at it. A screw is everything else: a lookup, a filter, a constraint check, a calculation, a sort. Work where a rule returns the same answer on the thousandth call as it did on the first. Sections 2 and 3 showed what that work costs on the wrong substrate. A function on a CPU settles it for a fraction of a penny and settles it the same way every time. A model settles it by reasoning, at GPU prices, and might not settle it the same way twice. The capex in Section 4 is the sound of an industry hammering screws.

Here is what makes this a choice and not a constraint. The line between the nail and the screw is not drawn for you by the silicon, and you do not unlock it by renting a larger model. You draw it. You look at a workload, you separate the parts that need judgement from the parts a rule settles, and you send each to the thing built for it. That decision is available today, to anyone, for nothing. The companies in Section 4 are not short of the tool. They are short of the discipline to point it at the right work, and the bill is the difference between the two.

There is a second saving folded inside the first. When the work genuinely is a nail, swing once and swing clean. A model handed a few hundred tokens of exactly what it needs is cheaper and sharper than the same model handed thousands of rows it has to wade through to find them. Less is more here in the plain sense: the smaller the input, the lower the cost and the better the answer. Discipline pays twice. You fire the model less often, and the times you do fire it, the call is short.

Put those together and a picture should be forming, the one I have been building since the chips. Look at almost any system running on AI today and the line is there to be seen: a thin slice of real judgement, wrapped in a great deal of deterministic work that was handed to a probabilistic engine because that was easier than thinking about it. The cost in Section 4 is not the price of intelligence. It is the price of refusing to draw that line.

That has all been principle, and a line you can see in the abstract is easy to nod at and hard to act on. In the next section I take a real operation, the kind that moves physical goods against a deadline, and I draw the line straight through it: what the model must never touch, what only the model can do, and what it costs when the boundary is in the wrong place.

Section 6

Example Scenario

I have made a lot of strong claims about architecture. A line you can see in the abstract is easy to nod at. So here is a real shape of business, and the line drawn straight through it.

For this I am going to use a made-up distribution company in the UK. Distribution and logistics is a data-heavy industry, and it takes a great deal of calculation to get a part from a shelf to a customer before they open. Nothing here describes a real operator. The numbers are representative of the sector, not drawn from a single firm.

The company turns over about a hundred million pounds a year. Its fleet runs to around a hundred and twenty vehicles, lorries for the trunk routes and vans for the local drops, working between fourteen depots and a central hub and moving close to ten thousand part-orders a day to shops, garages, and warehouses across the country. Orders arrive through the day. The lorries roll just after midnight, because the business lives or dies on a part landing before the customer opens. Not every order can ride on every lorry. Some parts are temperature-controlled, some are dangerous goods that cannot share a trailer with the wrong neighbour, weight has to sit correctly across the axles, and a next-morning order outranks a standard one. Today a mix of an ageing warehouse system and experienced people decides what goes where. It mostly works. It does not scale.

So they decide to automate, and they want to do it with AI. There are two ways this goes. Here is the first.

6.1 Route One: The Frontier Wrapper

They have read that AI is good now, that one frontier model is sharper than the rest, and they reach out to an agency that says it can solve the whole thing. The agency builds a demo. In a meeting, against a clean sample database, it works. Every order they throw at it routes correctly, in front of the room, every time. The contract is signed on the strength of that.

What gets built I will call the frontier wrapper: the frontier model wired straight into the live database with query tools and asked to allocate the night’s orders to the night’s lorries. No deterministic layer sits between the data and the model. The orders come in, the model reasons over them, the lorries get loaded. There is nothing wrong with how it is built. The model is capable and the queries are correct. What is missing is not quality. It is a layer.

For a while it holds. Then comes a night like this one.

A customer in London has had a rare part fail in one of their factories. A whole line is down, and it stays down until the part is replaced, costing them around fifty thousand pounds an hour in lost output. They order the replacement under an SLA that promises delivery within twelve hours: on site in London by eight the next morning. The distributor holds exactly one of that part. One unit, the only stock.

That same night the distributor is part-fulfilling an order of the very same part to a different customer, a new factory build up in Edinburgh, fifteen units against a far larger standing order. So the night’s manifest carries two demands for one part. A single urgent unit for London, buried in among everything else moving that night, and a large, loud, repeated line for Edinburgh that appears again and again across the run.

Here is the thing that breaks it, and it is worth understanding properly, because it is not what most people assume. The model is not too stupid. It is shown too much.

A model does not read a long input evenly. Hand it a short, clean input and it holds all of it. Hand it thousands of lines and it holds the beginning and the end and loses the middle. This is a documented failure mode in the research, and it is not a quirk of one model; it is how these systems behave when the context they are given is too large. Call it the lost-in-the-middle problem.

To see it rather than take my word for it, I will ask you something. Can you remember the total processing latency of the Mistral 7B model from Section 3? Not the time to the first token. The time to the finished answer. You will remember that there was an example, and roughly where it sat, and you have read every word in between. But the number itself has very likely gone, and to retrieve it you would have to scroll back. The answer was seven hundred milliseconds. You did not fail. You did exactly what anyone does with a long enough body of information: you kept the shape and the edges and let the middle go. The model does the same thing, for the same reason, and it does it to the night’s manifest.

The whole night’s orders go into the frontier wrapper. The Edinburgh line is large and it repeats. The single London unit sits in the middle of all of it. The model loses the middle, and with no priority layer in front of it, nothing in the system says London is an SLA-bound emergency that outranks a routine bulk order. So the model does the obvious thing with what it can still see clearly. It sends the stock to Scotland. All of it, the one reserved unit included, loaded onto the Edinburgh run.

The lorries load from midnight, ten or fifteen minutes apart. The Scotland-bound lorries carry the heaviest load and leave last, rolling out around two o’clock. From there it is six to seven hours on the road for a single driver, less for a double-manned team, with the legally required break somewhere in the middle. By eight in the morning the part is in Scotland, or close to it. London’s deadline does not fail at eight. It failed at two, the moment that lorry pulled out, and nobody knew, because there was nothing watching for it. The breach happened inside a context window no one can see.

It surfaces when the London customer rings at opening to ask where their part is. The line has been down all night at fifty thousand pounds an hour. The one unit is three hundred miles up the motorway on a lorry heading the wrong way, and there is no second unit to send instead. The SLA is breached, the customer is counting their losses, and the distributor cannot give them a straight answer about what went wrong, because the decision was made by a probabilistic engine reasoning over an input nobody can reconstruct. There is no dispatcher to have a word with. You trusted the AI, and there is no one to hold to account.

The cost arrives in two parts. The first is the running cost, and it is the smaller one. Pulling the whole night’s manifest through the model on every allocation decision means paying for every token of it, every time, all night. Those bloated calls add up, and on a busy night the company is paying tens of thousands of pounds to run the orders it could have run for a fraction of that. The second is the failure, and it does not fit on an invoice so neatly. The immediate scramble has a price: an emergency flight to carry the part from Edinburgh back down to London runs to something like eight thousand pounds, and even then it arrives hours after the SLA promised it. That is the cost you can write down. The one you cannot is the contract. A customer who pays the distributor around five million pounds a year across all its fulfilment work is now deciding whether to keep buying from a supplier whose system sent their emergency part to the wrong end of the country.

The model did not fail. It did exactly what a model does when it is shown too much. The architecture failed, and what was missing was never a smarter model or a tighter query. It was a layer that decided, before the model ever saw the orders, that one unit of that part was already spoken for.

Now rewind, and build it the other way.

6.2 Route Two: Deterministic First

Same company, same part, same night, same two orders competing for one unit of stock. This time a layer of deterministic code sits in front of the model and does the work that code is good at, the work that does not need judgement and must never be guessed.

The first thing that code does is sort by priority, before anything is allocated to anything. An SLA-bound emergency outranks a routine bulk order; that is a rule, not an opinion, and a rule is something code can hold without ever being wrong about it. The London order is flagged urgent and rises to the top. The single unit of stock is allocated to London and that allocation is settled, written down, locked, at one minute past midnight. The Edinburgh order is large and important and it is also not on fire, so it waits for the next available stock. None of this involves the model. None of it can hallucinate, because none of it is a judgement. It is a sort and an assignment, true or false, done.

Then code checks which lorries can actually honour the London delivery. The travel-time calculation that decided slot windows in the field-service build does the same job here: which vehicles can leave in time and reach London by eight in the morning. Code rules out everything that cannot. Wrong region, out. No capacity, out. Temperature or hazard mismatch with the load, out. Over axle weight, out. None of these are opinions either. Of the fifteen lorries loading that night, most are ruled out in code before the model is involved at all, and what remains is a short list of lorries, every one of which is already a valid answer.

Only now does the model run, and its job is small. It is not a frontier model wired into the whole database. It is a fine-tuned order-picking model, purpose-built for this one task and trained on thousands of past allocations. It is handed one order and a short list of valid lorries, a few hundred tokens of clean and relevant context, and asked the one genuine judgement call left in the problem: of the lorries that all work, which is the best fit on the soft preferences a human dispatcher would weigh. It picks. When it gets that wrong, the mistake is which of several valid lorries carried the part, not whether the part reached the right city. The city was settled in code, hours before, and the model never had the power to undo it.

This is the whole argument made physical. You only let the model make the choice that is actually a choice. By the time the data reaches it, the misroute that sent the part to Scotland is not unlikely. It is impossible, because no Scotland-bound lorry was ever on the list, and the one unit was committed to London before the model drew breath. The model cannot lose the middle of a long input, because it is never handed a long input. The expensive failure was not solved by making the model better. It was designed out, by deciding in cheap deterministic code everything the model did not need to touch.

That is what it means for the model to be the smallest part of the stack. Not the absence of AI, but AI placed exactly where it earns its cost and nowhere else. The wrapper made the model the whole system and inherited every weakness the model has. The substrate makes it one bounded component inside an architecture that does not depend on it being right about anything the code could have settled first.

Which opens the question Section 7 takes up. Once the model’s job is this small, once it only ever sees one order and a handful of valid options, a few hundred tokens at a time, why are you renting a frontier model to run it at all?

Section 7

What Comes Next

Section 6 ended on a question. Once the model only ever sees one order and a short list of valid options, a few hundred tokens of clean context at a time, why are you renting a frontier model to run it at all?

You are not, in any sense that matters. A model with a job that small does not need to live in someone else’s data centre. It does not need an army of rented accelerators warmed up and waiting. The work fits on hardware a company can own outright and run in the corner of its own building, behind the same locked door as everything else the business guards. The frontier wrapper had to ship every order out to a model it rented by the token. The bounded model never has to leave the premises.

That changes more than the bill, and the things it changes are the things that decide whether a regulated business can use AI at all. When the model runs on infrastructure you own, your data never leaves it. Nothing is handed to a third party to be reasoned over and handed back. For a company operating in the EU this is not a convenience; it is the line between being able to automate and not. The data stays resident where the law requires it to stay. It never crosses a border it is not allowed to cross, never lands in a region no one can name to a regulator, never sits inside a company in another jurisdiction whose government can compel access to it. The whole operation runs inside your own firewall, under your own security, on machines governed by people who answer to you. The tightest data-protection regime in the world stops being an obstacle to automation, because compliance and automation are no longer pulling against each other. The thing the model needs to see never has to leave the room.

There is a second kind of safety here, and it is worth holding apart from the first, because it comes from a different part of the stack entirely. A frontier agent wired into the open world carries an enormous attack surface. It reads whatever it is handed, and an instruction smuggled inside what looks like ordinary data can turn it against the system it was built to serve. This is a real and unsolved problem for agents that take the whole world as their input, and the larger and more capable the agent, the more there is to subvert. The architecture this paper has described barely exposes a surface to attack. The model never sees the open input. It sees a few hundred tokens that deterministic code has already chosen, shaped, and bounded. There is almost nothing for a malicious prompt to take hold of, because the model was never handed the room to be hijacked in. That protection has nothing to do with owning the hardware and everything to do with how small the model’s job was made. Owning the silicon keeps your data yours. Bounding the context keeps the model hard to turn.

No one has really built this yet. The companies bolting frontier models onto their operations are not building it; they have wired the model into everything and inherited every weakness it has. The legacy vendors are not building it; they cannot move. What this paper has described is not a product on a shelf. It is the shape of the next generation of software companies: a small model doing the one job that genuinely needs judgement, deterministic code carrying everything else, the whole of it running on infrastructure the business owns and controls, cheap to run and hard to break, compliant in the jurisdictions that matter because the data was never sent anywhere it should not go.

The founders who build that wave will not be the ones who command the most compute. They will be the ones who understood, before they wrote a line, where each piece of their computation was going to physically run, and routed every piece of it to the silicon that runs it best. That was the argument the whole way down. The model was never meant to be the system. It was always meant to be the smallest part of it.

Bibliography

References

  1. Knight, T. (2026). Why the Model Should Be the Smallest Part of Your Stack (Thesis 1). THK Digital. thkdigital.com/research/infrastructure-first-ai

  2. Hetzner. Cloud — General Purpose pricing and Price adjustment (new prices effective 15 June 2026). hetzner.com · docs.hetzner.com

  3. Amazon Web Services. EC2 On-Demand Pricing (m7i.large). aws.amazon.com · instances.vantage.sh

  4. Katz, J. (2024). pgvector performance: a 150x speedup. jkatz05.com

  5. AMD (2024, 10 October). 5th Gen EPYC (“Turin”, EPYC 9005) — AI inference positioning and launch. amd.com/inference · amd.com/newsroom

  6. GPU hourly rental trackers (NVIDIA H100), 2026. getdeploying.com · thundercompute.com · spheron.network

  7. GPU shortage 2026 (lead times) and foundry allocation status, Q1 2026. spheron.network · siliconanalysts.com

  8. Google Cloud, The Register, and Anthropic. Ironwood (TPU v7) general availability, and Anthropic TPU access. theregister.com · cloud.google.com · anthropic.com

  9. Jouppi, N. P., et al. (2023). TPU v4: An optically reconfigurable supercomputer for machine learning (ISCA 2023). arXiv:2304.01433. arxiv.org/abs/2304.01433

  10. OVHcloud, TrendForce, and The Register. Memory shortage and cloud pricing, 2025–2026. blog.ovhcloud.com · trendforce.com · theregister.com

  11. NVIDIA. GeForce RTX 5090 (MSRP “from $1,999”). nvidia.com Street pricing: author’s market observation, 7 June 2026.

  12. Branchaud, J. Speeding up an expensive PostgreSQL query: B-tree vs BRIN. dev.to

  13. Malkov, Yu. A., & Yashunin, D. A. (2020). Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4), 824–836. DOI 10.1109/TPAMI.2018.2889473. Preprint arXiv:1603.09320 (2016). arxiv.org/abs/1603.09320

  14. Steezr. pgvector 0.8 vs a dedicated vector DB: when Postgres is enough. steezr.com

  15. pgvector v0.8.0 release notes; ACORN predicate-aware ANN search (arXiv:2403.04871); dbi-services pgvector index guide (March 2026). github.com/pgvector · arxiv.org/abs/2403.04871 · dbi-services.com

  16. pgvector v0.7.0 release notes; Intel AVX-512 instructions. github.com/pgvector · intel.com

  17. Meta Engineering (2025, 8 May). Accelerating GPU indexes in FAISS with NVIDIA cuVS. engineering.fb.com

  18. Baseten. Benchmarking fast Mistral 7B inference. baseten.co

  19. Morgan Stanley. AI capex: a $740 billion banking opportunity. morganstanley.com

  20. Fortune (2026, 26 May). Uber COO on AI spending, tokens, and Claude Code (also reports Microsoft’s direct-licence cancellation; originating scoop The Verge, 14 May 2026). fortune.com · aimagazine.com

  21. Axios (2026, 28 May). AI spending, ROI, and enterprise costs. axios.com

  22. Epoch AI, LLM inference price trends; CloudZero, State of AI costs. epoch.ai · cloudzero.com

  23. Gartner (2026, 25 March). By 2030, inference on a 1-trillion-parameter LLM will cost GenAI providers over 90% less than in 2025. gartner.com

  24. AICC, via the National Law Review. Enterprise token-cost report: 2B+ API calls, tiered vs frontier-only routing. natlawreview.com

  25. Goldman Sachs; PJM Market Monitor (Monitoring Analytics); Consumer Reports. Electricity prices, data-centre demand, and household bills, 2025–2026. gspublishing.com · monitoringanalytics.com · consumerreports.org

  26. Ratepayer Protection Pledge (March 2026): Federal Register; The White House; CNBC. federalregister.gov · whitehouse.gov · cnbc.com