Rate a control on design, operating effectiveness, sample deviations, population coverage and automation, and get an audit-style conclusion.
Design and operating effectiveness are separate questions and are weighted separately: a well-designed control that fails in practice and a diligently operated control that addresses the wrong risk both score badly, for different reasons. The rule of three is shown because a clean sample of thirty does not prove a control never fails — it only bounds the failure rate at roughly ten percent, which is often a surprise to whoever chose the sample size. The residual risk multiplier is the direct link from a testing result to a risk register: an 80% effective control leaves a fifth of the inherent risk, and that is the figure that belongs in the register rather than the word effective. This is a management estimate, not an audit opinion.
Control Effectiveness
Effectiveness = 0.30 × design + 0.40 × operating effectiveness from the sample + 0.15 × population coverage + 0.10 × automation + 0.05 × evidence quality, all on 0–100.
Rule of three
With zero deviations, the rule of three gives an approximate 95% upper bound on the true error rate of 3 ÷ sample size.
Effectiveness = 0.30 × design + 0.40 × operating effectiveness from the sample + 0.15 × population coverage + 0.10 × automation + 0.05 × evidence quality, all on 0–100. Design and operating effectiveness are separate questions and are weighted separately: a well-designed control that fails in practice and a diligently operated control that addresses the wrong risk both score badly, for different reasons. The rule of three is shown because a clean sample of thirty does not prove a control never fails — it only bounds the failure rate at roughly ten percent, which is often a surprise to whoever chose the sample size.
The residual risk multiplier is the direct link from a testing result to a risk register: an 80% effective control leaves a fifth of the inherent risk, and that is the figure that belongs in the register rather than the word effective. This is a management estimate, not an audit opinion.
This calculator takes 6 inputs: Design assessment, Items tested, Deviations found in the sample, Share of the population the control actually covers, Automation level, Evidence quality. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
Sometimes, but not by arithmetic. You have to understand the deviation — a one-off caused by a departed employee is different from a systematic gap in the process — and either extend the sample or conclude a deficiency. What you cannot do is average it against successes and carry on.
Because a control that operates perfectly on the systems it monitors tells you nothing about the systems it does not. Coverage is where most control estates actually fail: the process is sound, and a third of the estate was never in scope.