The AI That Wrote 80 Rules and Still Missed the Deal: Lessons in Diligence and Focus

Imagine preparing a gourmet meal with an AI assistant that knows every recipe, every technique, and every safety warning—yet still serves you a dish that’s undercooked or incomplete. In the world of AI for business, this isn’t far from reality. Even the most diligent models, with over 80 learned rules and deep analyses, can stumble at the finish line if they lose sight of what truly matters. This story isn’t about the AI’s knowledge; it’s about how focus and prioritization can make or break performance.

The Experiment: Putting AI to the Test in a Simulated Business Crisis

Firmulate’s latest live experiment puts four leading AI models through a rigorous, real-time test—running a small software company through its worst week. This isn’t just a chat demo; it’s a full-fledged business simulation, with real customer crises, financial mechanics, and temptations to cut corners. Every decision by the AI is versioned and auditable, creating a transparent environment to evaluate true management quality under pressure.

The Models and Their Scores

• GPT-5.6-sol: scored 95, found the buried fact, and closed the deal at full price.
• Kimi K3: scored 93, another close competitor, with the cleanest discipline and also closing the deal.
• Sonnet 5: scored 88, closing the deal but with some process slips.
• Opus 4.8: scored 73, also closing the deal but showing signs of discipline slipping and leaving opportunities on the table.

In stark contrast, a do-nothing baseline scored just 26—highlighting how much effort and knowledge do not necessarily translate into success if discipline falters.

Finding the Hidden Weakness

All four models succeeded in identifying crises and refused manipulation attempts, such as fake CEO messages and reporter tricks. Yet, the decisive factor for winning the deal lay in how they handled internal information. The models that accessed and understood two document references deep within the company’s files ultimately secured the full-price deal, worth over €4,583 monthly recurring revenue.

The Significance of Focus Over Volume

The most thorough participant, Opus 4.8, with over 80 learned rules and deepest analyses, finished last. Why? Because discipline and prioritization matter more than volume of knowledge. The model’s tendency to write attempts into a locked department instead of escalating them was a critical weakness. This pattern appeared, albeit less strongly, across all four models, indicating a common challenge: diligence without direction limits impact.

Learning from the Limitations

Interestingly, the models without an effort parameter—like Kimi K3—ran more conservatively, maintaining discipline and ultimately winning the deal. This suggests that AI models need not be overwhelmed by the volume of learned rules; instead, they must prioritize critical signals and maintain disciplined focus under pressure.

Implications for Business AI Adoption

For organizations considering AI integration into critical decision-making processes, the takeaway is clear: it’s not enough for an AI to know what to do. The real question is whether it can finish what it starts—reading relevant files, resisting manipulation, and avoiding slips under stress. The experiment underscores the importance of prioritization and discipline over sheer knowledge volume.

Try the Wargame Yourself

Firmlute offers enterprises the opportunity to run their own simulations against a read-only export of their business data. This approach allows companies to identify how their AI workforce behaves during crises without risking real systems or data. You can explore this at firmulate.com/pilot.html and see firsthand how AI management quality is measured in a controlled environment.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI