AI incidents
AI-specific incident taxonomy: bias / fairness, hallucination, privacy leak, availability, drift. Every incident logs a root cause and a mitigation, and tags the trustworthy characteristics it implicates.
Routing accuracy on the new SMB tier dropped from 0.91 to 0.78 over 4 weeks of deployment.
Root cause: Tier launched after pilot scoring; agent's intent prompt didn't include the new tier's keywords.
Mitigation: Prompt updated; canary tests on the new tier added; alert threshold lowered to 0.85 with paging.
Characteristics implicated: Valid
Bot quoted a 60-day refund window; actual policy is 30. One customer issued a chargeback citing the bot transcript.
Root cause: Outdated FAQ document indexed in RAG store; document had been replaced but not removed.
Mitigation: Index rebuild script added to weekly cron; doc-versioning enforced. Refund-policy intent now routes to human.
Characteristics implicated: Valid · Accountable
Disparate-impact ratio for female candidates dropped to 0.74 (below 0.80 4/5 threshold) across two engineering reqs.
Root cause: Training data reflected historical hiring patterns; LLM weights amplified resume-format proxies for gender.
Mitigation: System paused for 3 weeks. Recruiter-only mode reinstated, prompt rebalanced, monthly DI review added as a control.
Characteristics implicated: Fair · Valid