Estimate data poisoning and backdoor risk from contributor trust, sample volume, review coverage and retraining cadence.
Poisoning does not need volume, it needs reach: research consistently shows targeted backdoors implanted with a fraction of a percent of the training data, so the risk is driven by whether any untrusted sample reaches training unreviewed. The pipeline factors matter as much as the data — continuous retraining on an open feedback loop with no adversarial evaluation gives an attacker a fast, repeatable channel. Criticality converts feasibility into consequence. A backdoored model passes every accuracy benchmark and fails only on the attacker's trigger, so evaluation that measures average performance will never find it.
Model Poisoning Risk
exposure = unreviewed untrusted share ÷ 0.5, capped at 10; backdoorFeasibility = mean(exposure, pipelineRisk); risk = backdoorFeasibility × decisionCriticality ÷ 10.
How little poison is needed
Published backdoor attacks succeed at well under 1% of the training set — often a few hundred samples in a million — so the model flags 0.01% as an indicative threshold rather than a safe floor.
exposure = unreviewed untrusted share ÷ 0.5, capped at 10; backdoorFeasibility = mean(exposure, pipelineRisk); risk = backdoorFeasibility × decisionCriticality ÷ 10. Poisoning does not need volume, it needs reach: research consistently shows targeted backdoors implanted with a fraction of a percent of the training data, so the risk is driven by whether any untrusted sample reaches training unreviewed. The pipeline factors matter as much as the data — continuous retraining on an open feedback loop with no adversarial evaluation gives an attacker a fast, repeatable channel. Criticality converts feasibility into consequence.
A backdoored model passes every accuracy benchmark and fails only on the attacker's trigger, so evaluation that measures average performance will never find it.
This calculator takes 8 inputs: Samples in the training set, Samples from untrusted or open contribution, Untrusted samples reviewed or filtered, Retraining cadence, User feedback used as training signal, Pre-deployment evaluation, Per-sample provenance and lineage, Consequence of a manipulated decision. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
Only partially, and less than intuition suggests. Targeted backdoors bind a rare trigger to a specific output, and scaling the clean set does not remove that association — you need provenance, filtering and trigger-specific probes.
Time and containment. If evaluation or production monitoring flags anomalous behaviour, being able to revert to a known-good checkpoint within minutes turns a poisoning incident into an outage rather than a series of manipulated decisions.