Model Poisoning Risk Calculator

Estimate data poisoning and backdoor risk from contributor trust, sample volume, review coverage and retraining cadence.

About Model Poisoning Risk Calculator

Poisoning does not need volume, it needs reach: research consistently shows targeted backdoors implanted with a fraction of a percent of the training data, so the risk is driven by whether any untrusted sample reaches training unreviewed. The pipeline factors matter as much as the data — continuous retraining on an open feedback loop with no adversarial evaluation gives an attacker a fast, repeatable channel. Criticality converts feasibility into consequence. A backdoored model passes every accuracy benchmark and fails only on the attacker's trigger, so evaluation that measures average performance will never find it.

Formulas

Model Poisoning Risk

exposure = unreviewed untrusted share ÷ 0.5, capped at 10; backdoorFeasibility = mean(exposure, pipelineRisk); risk = backdoorFeasibility × decisionCriticality ÷ 10.

unreviewed untrusted share: Untrusted samples that no human or filter checked, as a percentage
pipelineRisk: Retraining cadence, feedback loop, evaluation and provenance
decisionCriticality: Consequence of the model deciding wrongly on purpose

How little poison is needed

Published backdoor attacks succeed at well under 1% of the training set — often a few hundred samples in a million — so the model flags 0.01% as an indicative threshold rather than a safe floor.

samplesForBackdoor: Indicative sample count for a targeted trigger

Worked example

With the default values below, Model Poisoning Risk Calculator returns Poisoning Risk of Medium. Change any input above and every figure here is recalculated in your browser.

Inputs used

Samples in the training setaccepts 100–100,000,000,000 samples: 2,000,000 samples
Samples from untrusted or open contributionaccepts 0–100,000,000,000 samples: 180,000 samples
Untrusted samples reviewed or filteredaccepts 0–100 %: 25 %
Retraining cadenceone of Manual, gated release — 2, Scheduled monthly — 4, Weekly automated — 6, Continuous online learning — 9: Weekly automated — 6
User feedback used as training signalone of Not used for training — 1, Used after human curation — 4, Used automatically with rate limits — 7, Used automatically, unlimited — 10: Used automatically with rate limits — 7
Pre-deployment evaluationone of Held-out plus adversarial and backdoor probes — 2, Held-out benchmark suite — 5, Spot checks — 7, None — 9: Held-out benchmark suite — 5
Per-sample provenance and lineageone of Full lineage, reproducible builds — 2, Batch-level provenance — 6, None — 9: Batch-level provenance — 6
Consequence of a manipulated decisionone of Content ranking — 3, Fraud or abuse scoring — 7, Credit, hiring or medical triage — 9, Safety-critical control — 10: Fraud or abuse scoring — 7

Results

Poisoning Risk: Medium
Risk Score: 5.60 / 10
Untrusted Share of Training Data: 9.00%
Unreviewed Untrusted Share: 6.75%
Backdoor Feasibility: 8.00 / 10
Samples Typically Needed for a Trigger: 200 samples
Highest-Value Control: Curate user feedback before it becomes training signal — an open loop is an open poisoning channel

Step by step

1Values used
2Model Poisoning Risk
3How little poison is needed
4Poisoning Risk= Medium
5Risk Score= 5.60 / 10
6Untrusted Share of Training Data= 9.00
7Unreviewed Untrusted Share= 6.75
8Backdoor Feasibility= 8.00 / 10
9Samples Typically Needed for a Trigger= 200 samples

Assumptions and limits

The result is derived only from the 8 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax or compounding frequency into account.
Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

How is Model Poisoning Risk calculated?

exposure = unreviewed untrusted share ÷ 0.5, capped at 10; backdoorFeasibility = mean(exposure, pipelineRisk); risk = backdoorFeasibility × decisionCriticality ÷ 10. Poisoning does not need volume, it needs reach: research consistently shows targeted backdoors implanted with a fraction of a percent of the training data, so the risk is driven by whether any untrusted sample reaches training unreviewed. The pipeline factors matter as much as the data — continuous retraining on an open feedback loop with no adversarial evaluation gives an attacker a fast, repeatable channel. Criticality converts feasibility into consequence.

Why does Model Poisoning Risk matter?

A backdoored model passes every accuracy benchmark and fails only on the attacker's trigger, so evaluation that measures average performance will never find it.

What values do I need to enter?

This calculator takes 8 inputs: Samples in the training set, Samples from untrusted or open contribution, Untrusted samples reviewed or filtered, Retraining cadence, User feedback used as training signal, Pre-deployment evaluation, Per-sample provenance and lineage, Consequence of a manipulated decision. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.

Does more clean data dilute poisoning?

Only partially, and less than intuition suggests. Targeted backdoors bind a rare trigger to a specific output, and scaling the clean set does not remove that association — you need provenance, filtering and trigger-specific probes.

What does a rollback plan buy?

Time and containment. If evaluation or production monitoring flags anomalous behaviour, being able to revert to a known-good checkpoint within minutes turns a poisoning incident into an outage rather than a series of manipulated decisions.