The first thing this AI startup did after raising $9M? Make its own model weaker on purpose.
Four tiers weaker than a frontier model, to be exact.
Everyone Assumes "Smarter AI" Is the Answer
When AI gets something wrong, the default move is to wait for the next version — a bigger model. GPT struggling? Wait for the next GPT. Claude confused? Wait for the next Claude. "Once models get big and smart enough, they'll stop making mistakes" has basically been the industry's unspoken assumption.
But the numbers don't back that up. A 2026 benchmark of 37 models found overall hallucination rates ranging from 15% to 52%, and even the top-performing models still hovered around 15-17%. For legal queries, it jumps to 69-88%; for unguarded medical tasks, it can spike as high as 64%. Models keep getting better, and yet these numbers just won't budge.
And this isn't just a stats problem. Global losses from AI hallucinations hit $67 billion in 2024 alone, and in Q1 2026 flawed financial analysis wiped out $2.3 billion. Companies are now spending $14,200 per employee per year — 4.3 hours a week — just fact-checking AI outputs. And 47% of executives admit they've already made a major decision based on content that turned out to contain hallucinations.
So while everyone else waits around for "the next model to fix it," one startup — backed by a $9 million bet from a16z — went the opposite direction entirely.
Probably Went the Other Way
Peter Elias, who left his role leading the data platform at Optimizely to found Probably, didn't try to make the model bigger. Instead, he built a harness that watches the model. Elias calls it a "data science mech suit" — every answer the LLM produces gets checked against a deterministic verifier that cross-references real datasets, and the model gets retrained until it consistently passes that check.
The result is counter-intuitive: once the harness is doing its job, a model four tiers weaker than a frontier model can hit comparable reliability. Here's how Elias put it.
"The better the harness engineering, the weaker the model can be. If you can refine the context enough, the model doesn't have to work that hard."
— Peter Elias, founder of Probably
The target is 99.99% accuracy — the kind of number you'd normally only expect from a deterministic system. Why does this matter for the bottom line? Because a weaker model can run on local hardware (Beta 0.1 currently supports M1 through M5 Apple Silicon), which slashes token costs and lets customer data stay local while only the inference gets sent to the cloud. Probably says this setup cuts infrastructure costs by up to 25%.
Elias's take on why is worth noting too: "Big labs have no incentive to do this. The more times you have to re-run a model, the more money they make". a16z made a similar point in a recent investment thesis, arguing that "the companies that make AI boring will create the most value" — where "boring" doesn't mean uninteresting, it means predictable and trustworthy, like a database.
| Old Approach — Bigger Model | Probably's Approach — Verification Harness | |
|---|---|---|
| How to improve accuracy | Swap in a bigger, pricier frontier model | Filter out a weaker model's wrong answers with a deterministic verifier |
| Cost | Token and API costs keep climbing | Can run locally, cuts infra costs by up to 25% |
| Grading criteria | Another model does the grading (LLM-as-judge) | Human-defined rules cross-checked against real datasets |
| Limitation | Also gets applied to open-ended creative work with no "right answer" | Built specifically for tasks with a defined right answer (data, accounting, healthcare) |
That "grading criteria" row matters more than it looks. The trendy LLM-as-judge approach still relies on another probabilistic model doing the scoring, which means it's stuck with the same biases — position bias (favoring whichever answer comes first) and length bias (favoring longer answers). Probably doesn't hand that grading job to AI at all — it swaps it out for deterministic, human-defined rules. It's basically a statement: we're not going to ask another AI whether the AI got it right.
This Is Already Happening in Korea, Too
This isn't just a Silicon Valley story. Just yesterday (July 14, 2026), Korean AI company GenesisCoTechs AI commercialized its own take on the same problem: the 'AI Debate Engine'. The approach is different — where Probably verifies answers with deterministic rules, this platform pits ChatGPT, Claude, Gemini, and Grok against each other in a "compete → verify → collaborate" framework, with verification grounded in actual execution. But the underlying instinct is the same: don't just trust a single model's probabilistic answer.
Why does this matter for companies here, right now? Because the more AI gets embedded in real workflows, the more unverified risk comes along with it. A recent domestic security report flagged prompt injection as a critical vulnerability in enterprise AI operations, and found that 37% of large-enterprise vulnerabilities go unpatched for over a year. "How much can we trust what the AI just told us" isn't just a question for the tech team anymore — it's a question for every organization putting AI tools into production.
| Approach | Core Idea | Who's Doing It |
|---|---|---|
| Deterministic verification harness | Cross-check against a ground-truth dataset and hardcoded rules | Probably |
| Multi-model debate | Pit multiple LLMs against each other to reach consensus | GenesisCoTechs AI |
| LLM-as-judge | A separate LLM scores the answer | Most general-purpose evaluation frameworks |
All three approaches have their place, but the one your organization can realistically copy right now is the deterministic verification harness. You don't need to run multiple models simultaneously or train a separate grading model — you just need the core principle: define the right answer in code, and check the AI's answer against it.
How to Apply This to Your Own AI Workflow
You can't actually buy Probably yet — it's still in beta, and limited to Apple Silicon. But the principle it's proving — "wrap deterministic verification around any task that has a right answer" — applies to whatever AI workflow you're running today.
- Start with verifiable tasks
Pick tasks where "right or wrong" is clear — numbers, code, database lookups. This approach doesn't work for creative writing or open-ended questions. - Get a ground-truth dataset
You need something to check the AI's answer against. Already-verified historical data, internal databases, and official docs are your best candidates. - Build the verifier with deterministic rules
Don't hand the grading job to another AI. Write hardcoded rules — "pass if this condition holds, reject otherwise" — yourself, so the judging stays bias-free. - Use failure cases to refine your prompts and context
Collect the wrong-answer patterns your verifier catches, and keep refining your prompts and the context you feed the model. As Elias put it, "the better the context gets, the less hard the model has to work." - Keep a citation trail
Log what evidence each AI answer was verified against, so you can answer "why did we trust this?" later. Trust ultimately comes from that record.
Deep Dive Resources
TechCrunch — original report The first piece to cover Probably's $9M seed round and the "data science mech suit" concept techcrunch.com
a16z — Investing in LMArena a16z's investment thesis on the "reliability layer for AI" and why boring AI wins a16z.com
SQ Magazine — LLM Hallucination Statistics 2026 Hallucination rate data by domain, based on a 37-model benchmark sqmagazine.co.uk
Future AGI — LLM-as-a-Judge Guide How LLM grading works, and its failure modes like position and length bias futureagi.com
Tendem — The Real Cost of AI Hallucinations A report on losses and verification costs from AI hallucinations, based on enterprise data tendem.ai
GTT Korea — Prompt Injection Vulnerability Report Security risk statistics for enterprise AI operations from a Korean perspective gttkorea.com
