An automated system can make a mistake that is easy to undo, or one that affects money, access, privacy, safety, or a person’s ability to get help. The cost depends on the task and its context; a single headline number cannot describe every system.
Start with the action
Write down what the system is allowed to do, what information it uses, who may be affected, and which actions can be reversed. Separate an incorrect suggestion from an action that changes a record, sends a message, moves money, or grants access. The difference changes both the potential impact and the response.
Make assumptions visible
For each plausible failure, estimate the direct impact, the chance and frequency of the event, and the effort required to detect and recover. Include support and rework, as well as privacy, safety, and trust impacts that may not fit a simple financial calculation. Label estimates as estimates and show where evidence is weak.
For example, a team might compare a reversible draft with an automatically sent message. The first may need a correction before use; the second may reach many people and require follow-up. This is an illustrative comparison, not a measurement of any product or customer outcome.
Use the assessment to choose controls
Higher-impact actions may call for narrower permissions, review before execution, clearer confirmation, or a reliable way to pause and recover. Lower-impact tasks may need a different balance. The right controls depend on the system’s actual capabilities and the people who rely on it.
Revisit the assessment when the model, tools, data, audience, or operating conditions change. A useful risk estimate is an aid to a decision, not proof that a system is safe or ready to launch.