Designing AI Systems That Fail Honestly
The most dangerous AI systems do not crash. They keep responding while their evidence is stale, policies conflict, retries multiply, tool calls become expensive, or the system can no longer justify the commitments it is making.
In this hands-on workshop, participants redesign a failing AI-commerce workflow that appears technically healthy but is behaviorally unsafe. They work through conflicting inventory information, an ambiguous payment timeout, a stale return policy, and an exhausted AI tool-call budget. The objective is to design a system that remains useful without making promises it cannot defend.
Traditional reliability asks whether a service is up, fast, and error-free. AI systems introduce a higher bar: whether the system is still safe to trust. A dashboard can be green while customers receive incorrect answers, duplicate actions, unfair denials, or decisions nobody can explain later.
This workshop gives teams an architecture method for controlling that risk. Instead of treating the system as simply “working” or “down,” participants design modes that progressively reduce autonomy: answer normally, become conservative, defer the decision, freeze liability actions, or escalate to human review.
Session details:
Participants work in teams on a realistic commerce-agent scenario. The agent can answer customers, check inventory, interpret return policy, initiate refunds, and call external tools. Then the workshop injects failures:
Inventory data is eight minutes stale.
A payment call times out after the customer may already have been charged.
A retired return policy outranks the current policy.
Retries create duplicate side-effect risk.
The tool-call and cost budget is nearly exhausted.
APIs return successful responses, but the evidence is no longer reliable enough for a customer commitment.
For each failure, teams answer:
What promise has the business made to the customer?
What can the AI still safely show or say?
What evidence is admissible for this decision?
What must the system stop doing?
When should it defer, refuse, or escalate?
What proof is needed before automation can be re-enabled?
Participants use an architecture canvas to make the hidden controls explicit: customer promise, commitment boundary, admissible evidence, freshness rules, churn budgets, degraded modes, human-review triggers, and decision receipts.
About Rohit Bhardwaj
Rohit Bhardwaj is Director of AI & Data Architecture at Salesforce, an O’Reilly instructor, and founder of AI Architect Circle. With 30 years of experience in software and enterprise architecture, he helps engineers, architects, and technology leaders make architecture decisions they can explain and defend.
His work brings together enterprise AI, agentic systems, data architecture, and distributed systems. He focuses on the decisions that determine whether a system remains reliable, secure, accountable, and economically sustainable when it encounters real-world demands.
Rohit’s talks and workshops explore practical questions: What happens when an AI agent acts on conflicting information? Can a failed payment be retried safely? Who can stop an autonomous action, and what evidence explains the decision afterward? His teaching connects these questions to architecture, customer trust, and business outcomes.
He is the author of System Design with AI Interview Guide: Designing Scalable, Agentic, and Defensible Systems, published by Apress. He also developed ARCHVIEW™, ROCS Loops™, and LLM-SHIELD™ to help teams structure architectural thinking and address the challenges of building trustworthy AI systems.
Through AI Architect Circle, Rohit extends this learning through live discussions, book walkthroughs, videos, and interactive architecture challenges. These challenges explore incidents such as payment timeouts and conflicting bookings, helping participants practice the judgment behind their design decisions.
Continue learning with Rohit
Join Rohit live on Saturdays at 9:00 a.m. Eastern
System Design with AI — Live with the Author is a free live podcast and Q&A series featuring practical system-design scenarios, AI architecture discussions, and questions from the community.
See the next session and register
Try an interactive architecture challenge
Put your design decisions under pressure and explore the consequences.
Explore the challenges
Explore AI Architect Circle
Find architecture lessons, videos, resources, and ways to continue learning.
Visit AIArchitectCircle.com
Read the book: Amazon · O’Reilly
Watch and connect: YouTube · Instagram: @aiarchitectcircle · LinkedIn · X / Twitter: @rbhardwaj1
More About Rohit »