AI Model Security Risk Calculator

likelihood = mean(exposure, guardrail weakness, provenance risk, monitoring gap); impact = mean(data sensitivity, tool access, output-handling risk); risk = likelihood × impact ÷ 10, reduced 15% where a human approves consequential actions. The two factors that dominate real AI incidents are what the model can do and how its output is consumed — OWASP's excessive agency and improper output handling. A chatbot that returns escaped text with no tools is a content problem; the same model wired to write actions and a shell is a remote-code-execution surface reachable by anyone who can phrase a sentence. Guardrails reduce likelihood but never to zero, because natural language has no parser to sanitise. This is a prioritisation estimate, not assurance.

Every AI security control eventually reduces to the same question — if the model is fully manipulated, what can it actually reach — and that is a design decision, not a filter setting.

This calculator takes 8 inputs: Sensitivity of data in context or retrieval, Who can reach the model, Tools and actions the model can invoke, How output is consumed downstream, Input and output guardrails, Model and data provenance, Logging of prompts, outputs and tool calls, Human in the loop for consequential actions. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.

Because prompt filtering is heuristic. There is no formal grammar separating instruction from data in natural language, so guardrails raise the cost of an attack rather than eliminating the class. Design assuming they will eventually be bypassed.

It helps materially, which is why the model reduces the score — but approval fatigue is real. A human clicking approve on the two-hundredth request of the day is not a control, so keep the volume of consequential actions low enough that review stays meaningful.