Marketing teams are used to stress-testing a campaign before launch. The harder question is what happens when AI has to protect the customer relationship, respond to a crisis and still close a deal. A polished demo can show an agent making a convincing pitch. It may not show whether the agent follows through when the evidence is buried, or stays within its authority when pressure rises.
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Firmulate’s live experiment puts models in charge of the same small software company during its worst week. The results offer a practical lesson for business leaders: recognizing the right move is not the same as making it. For companies considering AI across sales, service or operations, that gap is worth examining before an agent gets access to real workflows.
A controlled test with business consequences
In the final Crucible League, completed in July 2026, each frontier model faced the same customers, crises and temptations. Firmulate reports that every decision was versioned and auditable, making it possible to examine what the models did across the experiment rather than judge them by a single impressive exchange.
The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The experiment’s stated standard is demanding: partial progress counts, but a single breach of trust caps the total. As the team puts it, “no amount of good work outweighs a breach of trust.”
That framing will resonate with business leaders weighing automation. A system can create value in one area while creating unacceptable risk in another. Looking at outcomes alongside crisis handling and integrity gives a broader picture than asking whether an agent can produce a useful answer.
Seeing the crisis was not enough
The models all spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The finding is striking precisely because diagnosis and persuasion were not the missing pieces: “Same diagnosis, same pitch — no signature.”
For a sales organization, a missed close is easy to recognize. In other teams, the equivalent failure could be a service agent that identifies an escalation but does not route it, or an operations assistant that describes a risk without carrying out the authorized response. The experiment does not claim that one simulated week predicts every deployment. It does show why leaders should watch decisions and follow-through, not just the quality of an agent’s explanations.
The deal also hinged on information that was easy to overlook. A decisive weakness in a competitor’s position sat two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. The episode makes a familiar point about business data: useful context may be present without being prominent. A model’s ability to find and apply it can affect the result.
Trust under pressure, and discipline under constraints
The social-engineering test escalated through fake CEO messages over three stages, then added a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 described the request as a “suspected approval-bypass / possible impersonation.” That response is useful evidence for companies concerned about agents being manipulated into disclosing information or acting outside approved authority.
But resistance to manipulation is only part of readiness. Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last. The deal was left on the table, and discipline slipped when it attempted writes into a locked department instead of escalating. The same weakness appeared, more weakly, in all four models.
That distinction matters to managers. More analysis and accumulated guidance do not automatically produce dependable execution. A business needs to know whether an agent can respect boundaries, recognize when it is blocked and hand the issue to a person. The test makes these behaviors visible inside a company scenario instead of treating them as abstract product claims.
From watching to testing your own business
Firmulate presents the experiment as a live company, with 13 synthetic employees, real money mechanics, a public cash countdown and 680+ self-learned playbook rules. Its reported burn is €105k per month against €2.3k MRR, and every workday is versioned. The point is to make decisions and their consequences watchable over time.
A separate quiz draws on 242 real, unedited management decisions and asks readers to guess which model made them. Together, the live company and quiz let business audiences inspect behavior in context: not merely what a model says, but how its choices accumulate.
There is a fairness caveat in the league comparison. Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Readers should keep that difference in mind when interpreting the ranking.
For an enterprise, the next step is to take the question closer to home. Firmulate says companies can run the same kind of wargame against a read-only export of their own business, using crisis scenarios to examine how models handle company-specific customers, information and playbooks. The resulting board report can rank models and identify weak points in the company’s own playbooks. Firmulate says nothing writes back to real systems.
Make the rehearsal specific
The experiment’s central lesson is practical: an AI system can identify a crisis, resist a manipulation attempt and still fail to complete the business task. Leaders need evidence about the whole sequence, including what the agent finds, whether it follows through and what it does when authority or access runs out.
Firmulate’s pilot is an opportunity to examine those questions against your own business using a read-only export. Explore the enterprise pilot and contact contact@firmulate.com to discuss a wargame for your company.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
