Enterprise AI reliability needs a broader case than benchmark accuracy. The relevant question is whether the system performs dependably over a stated period and under stated operating conditions, including its retrieval, memory, tools, permissions, oversight, system interactions and reasoning process.
That case should cover more than component errors. The proposed diagnostic framework distinguishes technical-component, operational-loop, agentic-conduct, and network or governance failures. It also links reliability work to SMART, the NIST AI Risk Management Framework, TEVV, sequential monitoring and FRACAS.
Evidence must match the deployed configuration. Changes to prompts, policies, models, tools, skills or workflows should be documented, with affected components retested because earlier evidence may no longer apply. For agents, monitoring should inspect actions before external effects and route tool calls, credential requests, state changes and external messages through an independent policy-enforcement point.
A usable reliability record connects the task and system configuration to exposure, retrieved information, tools, memory changes, permission checks, human interventions, handoffs, external effects, recovery outcomes, failure severity and corrective actions. These records can support analysis of recurrent events and whether a repair improved reliability.
The evidence base has important limits. Current controlled studies and incident registries do not support general AI-agent failure-rate estimates because many studies use simulated tasks or author-constructed faults. Longitudinal, change-controlled field data with operating context remain needed before controlled findings can be treated as general field-rate evidence.