Why grocery's AI bill doesn't have to scale with its business, and what happens to the cost curve once a retailer owns the intelligence instead of renting it.
Every grocery executive I sit across the table from this year asks some version of the same question. Not “should we do AI.” That debate ended sometime around when Walmart put Sparky inside ChatGPT and Kroger signed a national partnership with Google Cloud and Gemini. The question now is narrower, and harder: what does it actually cost to run this at scale, and who controls that cost curve?
That question deserves a real answer, with real numbers, not a vendor’s assurance that “it’ll get cheaper.” So I want to walk through the actual economics — where the savings come from, why they compound rather than erode, and why I think the next eighteen months will separate grocers who rent their intelligence from grocers who own it.
01 · THE HIDDEN COST CURVE
The hidden invoice
Start with the uncomfortable part. If you build a grocery AI experience directly on top of a general-purpose LLM — the path Walmart took with OpenAI, and the path Kroger is now taking with Gemini — you are not buying a fixed-cost platform. You are buying a meter. Every prompt, every reasoning step, every retry runs a token bill, and that bill scales linearly at best with your shopper adoption. The better your AI experience performs, the more it costs you to run it. That’s an inverted incentive for any retailer with a P&L to defend.
We instrumented this directly rather than take anyone’s word for it, our own included. In our Q1 2026 pilot with Giant Eagle, we logged the full token stream — prompt and completion — on both stacks in parallel, against the same shopper query, the same catalog scope, and the same response constraints.
Fourteen times isn’t a rounding error. At the scale a national or large regional grocer runs — tens of millions of AI-influenced sessions a year, growing every quarter as adoption climbs — the gap between those two cost curves is the gap between an AI program your CFO can underwrite with confidence and one that becomes a standing line-item risk on every board deck. We modeled this out to Year 3 for one regional partner at a projected ~125 million annual AI sessions. The cost avoidance versus a rented-LLM baseline lands around $3.8M a year. That’s not revenue we’re claiming credit for. That’s the invoice that never gets sent.
02 · WHY IT’S STRUCTURAL
Why the discount is structural, not promotional
Here’s the part that I think gets missed in most AI vendor pitches. Our cost advantage isn’t a subsidized rate we’re offering to win your business. It’s architectural, and it comes from where the intelligence is actually being sourced.
A general-purpose model has to reason over a massive store of data every time it’s asked a grocery question. It doesn’t know what’s on your shelf today, what your shopper bought last Tuesday, or what’s actually in stock at the store she’s walking into. So the burden of constraining, grounding, and personalizing that answer gets pushed into the prompt — which means more tokens, more reasoning steps, more retries, and a fatter, more unpredictable cost distribution.
Delectable inverts that order of operations. We build two proprietary knowledge graphs, seeded by your own first-party data.
Retrieval, ranking, filtering, and cart assembly all run deterministically against those graphs — built from your data, running on your tenancy. The general-purpose LLM only gets called at the very end, for final language formatting, after the data has already been filtered down to what’s actually true, in stock, and relevant to that household.
That’s the mechanism, plainly stated. Source the intelligence from data you already own, constrain the output to inventory that’s actually real, and call the expensive general-purpose model only for the narrow job it’s actually needed for. In our pilot instrumentation, that came out to zero off-catalog recommendations across thousands of sessions. Not a low rate. Zero.
03 · WHERE THE CURVE IS HEADED
From rented tokens to owned silicon
“I'd be doing this topic a disservice if I stopped at ‘we're cheaper today’ without addressing where the industry's headed, because I think we're at the cusp of a second shift most retail AI conversations haven't caught up to yet.”
04 · THE DEMAND SIDE
What this means for the consumer’s wallet
Cost efficiency on the backend only matters if it translates into value the shopper actually feels, so let me turn to the demand side, because the savings there are just as real and, I’d argue, even more durable.
The first generation of grocery AI — the Walmart-OpenAI, Kroger-Gemini, Instacart-everyone pattern — treats intelligence as a utility you meter and pay for per call, hosted on someone else’s cloud, trained in part on someone else’s aggregate data. That was a reasonable starting point when foundation models were the only path to a credible shopping assistant. It is increasingly not the only path, and for high-volume, narrow-domain applications like grocery, it’s not even the most economical one.
What I’m watching now, and what we’re building toward with our own architecture, is the migration of the deterministic, high-frequency parts of the reasoning stack onto smaller, fine-tuned, increasingly locally-hosted or owned-hardware models — with general-purpose frontier LLMs called only for the residual cases that genuinely require open-domain reasoning. This isn’t a contrarian bet; it’s the same maturation curve every compute-intensive industry has gone through. Enterprises don’t rent CPU cycles forever once the workload is large enough and predictable enough to justify owning the infrastructure. They build for it. Grocery AI, at the transaction volume of a 150-to-1,500-store chain, crosses that threshold faster than almost any other AI use case in retail, because the query pattern is narrow — a bounded catalog, a bounded set of households, a bounded set of intents — even as the volume is enormous.
The economic logic is straightforward once you see it. Every token you send to a rented, general-purpose model is a marginal cost that scales with your success. Every token you can instead resolve on a smaller model you’ve fine-tuned on your own catalog and household data — whether that’s hosted on your own cloud tenancy today or, over the medium term, on owned or reserved hardware — is a cost that starts to look more like depreciation than a utility bill. Retailers who build their AI program assuming the token meter is a permanent fact of life are underwriting a cost structure that only gets more expensive as they succeed. Retailers who build toward owned, narrow, fine-tuned models on top of their own first-party data are underwriting a cost structure that gets cheaper as they scale. That gap compounds every fiscal year.
Both of these — budgeting and pantry-awareness — are also, not coincidentally, the features that build the kind of trust that keeps a household loyal for years rather than one promotional cycle. Consumer economics and retailer economics point the same direction here, which is rare enough in retail AI that it’s worth calling out directly.
05 · THE SUPPLY SIDE
What this means for the grocer’s cost structure
Now to the side of the ledger that, frankly, gets less attention in most AI vendor conversations but matters just as much to a CFO: the operating-cost reductions inside the retailer’s own four walls.
Customer service load drops when the shopping experience stops generating the questions in the first place. A meaningful share of grocery e-commerce service volume traces back to friction in discovery and substitution — a shopper couldn’t find what she wanted, or an out-of-stock item got swapped for something that violated a dietary restriction she’d never have accepted. An agentic layer that reasons over allergen, dietary, and preference constraints before the substitution is ever offered removes that friction at the source.
06 · THE THREAD CONNECTING ALL OF IT
Three moves, one architecture
If there’s one idea I want an executive team to leave this article with, it’s this: the cheapest AI architecture and the most trustworthy one are the same architecture. Everything else follows from three decisions.
That’s not a coincidence, and it’s not one I take lightly, because it would have been very easy to build this company the other way — wrap a general-purpose model in a grocery-flavored prompt, ship fast, and let the token bill become tomorrow’s problem. We didn’t, because a system grounded in a retailer’s own catalog and a retailer’s own household data is, by construction, cheaper to run, faster to respond, harder to hallucinate with, and easier for a CFO and a CIO to sign off on in the same room. Own the data. Constrain the model. Call the expensive reasoning only when it’s genuinely needed. Everything else — the 14× cost advantage, the sub-second latency, the zero off-catalog recommendations, the compounding household intelligence that gets sharper with every basket — follows from that one architectural decision.
Grocery is the last major consumer category to make the shift to AI-native discovery.
The retailers who move first won’t just win on shopper experience. They’ll be the ones who built their AI program on a cost curve that bends down as they scale, while their competitors are still discovering what their token bill looks like at ten times today’s volume.