Work out guardrail false block rate instantly with clear inputs, formula shown and shareable results.
A guardrail has two error rates that must be tracked separately. The false block rate is wrongly blocked requests over all requests, which is what users experience as the product refusing reasonable work. Guardrail precision is the share of blocks that were justified. A guardrail blocking 2.4 percent of traffic with 15 percent of those wrong is denying service to 360 legitimate users per 100,000.
Guardrail error rates
false block rate = wrongly blocked / total requests; precision = (blocked - wrongly blocked) / blocked
It depends on the harm. For serious safety categories, accept a higher false block rate; for style or tone policies, precision matters more because over-blocking damages the product.
Human review of a random sample of blocks. Reviewing only appeals biases the estimate, because most wrongly blocked users never appeal.