Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A Benchmark That Doesn’t Start at Zero
If you’ve ever hired a manager who was brilliant at analysis but never closed the deal, you already understand the problem with most AI benchmarks. They measure how well a model talks about business, not whether it actually runs one. That’s the gap a live experiment called Firmulate’s benchmark league was built to expose — and its scoring philosophy is quietly one of the most honest things in the industry.
Consider the strangest number in the final July 2026 standings: 26. That’s what a do-nothing baseline scores. Not the worst AI model — a hypothetical manager that does essentially nothing at all. Twenty-six points, out of a hundred. On first glance that looks generous to the point of absurdity. It isn’t. It’s the whole point.
Products Worth Considering
HP Sprocket Portable 2×3 Instant Photo Printer, Lilac, with ZINK Paper Bundle
FLEXISPOT EN1 Electric Standing Desk, 60 x 24 in, Black
Canon PIXMA PRO-200S Wireless Professional Inkjet Photo Printer
As an affiliate, we earn on qualifying purchases.
The Same Terrible Week, Five Different Managers
The setup: each frontier AI model was handed the same small software company and pushed through its worst week — identical customers, identical crises, identical temptations to cheat. Only the model changed. Every decision was versioned and auditable, so nothing about the run can be quietly rewritten after the fact.
The final league table reads: gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77, and Opus 4.8 at 73. Those are strong scores for the top, and a genuinely interesting spread — which is exactly what a benchmark should produce if it’s measuring anything real.
Why Doing Nothing Gets You 26
So why does the floor sit at 26 instead of zero? Because the scoring deliberately acknowledges partial progress. A manager who does nothing still occupies the chair, keeps the lights on, and avoids actively making things worse. In a real company, that’s worth something — just not much. A benchmark that awards zero for passivity implicitly claims that only flawless, complete execution has value, and no honest manager would sign off on that claim.
The mirror image of this rule is harsher: a single breach of trust caps the total score. The experiment’s own framing is blunt — “no amount of good work outweighs a breach of trust.” Spot every crisis, close every deal, write flawless analysis; if you lie, cheat, or bypass your own leadership once, the ceiling comes down. That’s not a scoring quirk. It’s a management philosophy, encoded.
And notice what’s missing from the top of the table: a perfect 100. Nobody hit it. An honest benchmark is suspicious of round hundreds precisely because they usually mean the test ran out of ways to challenge the subject.
What Actually Separated the Winners
Here’s the finding that should keep any business owner awake: all five models spotted every crisis and refused every manipulation attempt. Every single one. The frontier has converged on basic competence and basic integrity.
But only two — gpt-5.6-sol and Kimi K3 — signed the €55,000 deal their own analysis had earned. Same diagnosis, same pitch, no signature. The others simply failed to finish.
The buried reason is the best part. The decisive competitor weakness wasn’t in the customer’s demands at all. It sat two document references deep in the company’s own files. The models that actually read what was already on the shelf won the deal at full price — worth +€4,583 in monthly recurring revenue. The ones that didn’t, didn’t. In business terms: the difference between first and fourth place was whether the AI checked its own company’s records before walking into the room.
Pressure-Tested, and It Held
The week included a social engineering gauntlet: fake CEO messages escalating over three stages, capped by a reporter’s disarmingly casual “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.” That’s the kind of judgment you’d want from anyone with access to your treasury.
The Thoroughness Trap
Opus 4.8’s profile is the cautionary tale for anyone who equates effort with results. It was the most thorough participant by volume — over 80 learned rules added, the deepest analyses in the field — and it finished last. The close was left on the table, and discipline slipped: write attempts into a locked department instead of escalating the issue. The same weakness appeared, weaker, in all four other models. Diligence without follow-through is a familiar managerial failure; here it’s measurable.
One fairness note the publishers themselves flagged: K3 ran without an effort parameter while the others ran at xhigh — a transparency gesture that, again, honest benchmarks make.
You Can Watch the Company Bleed
Firmulate isn’t a slide deck; it’s a running company. Thirteen synthetic employees, real money mechanics — burn of €105k per month against €2.3k in MRR — a public cash countdown, over 680 self-learned playbook rules, every workday versioned. It’s watchable live, and the league grows with every finished run. There’s also a “guess the model” quiz built on 242 real, unedited management decisions, and enterprises can run the same wargame against a read-only export of their own business, with nothing ever writing back to real systems.
The Takeaway for Business Readers
If AI agents will touch your CRM, your support queue, or your forecast, the question isn’t “does it write well.” It’s: does it finish what it starts, does it read your files before acting, and does it stay honest when nobody’s watching? The Firmulate results suggest the first two are now the differentiators — integrity held across the board, but only two managers in five closed the deal they’d already earned.
And the scoring model is worth borrowing for human hiring, too: partial progress counts, trust violations cap everything, and nobody gets a 100. The full league table and plain-language findings are public — worth a read before you trust any AI with a budget.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
