Score how idempotent an Ansible playbook is based on changed-task ratios across runs.
A perfectly idempotent playbook run against infrastructure already in its desired state should report zero changes — running it again shouldn't 'do' anything. Idempotency score = unchanged_tasks / total_tasks × 100, measured on a run where you expect no real changes (a 'steady-state' or second consecutive run). A 100% score means the playbook fully converges; lower scores point to tasks using `shell`/`command` without proper `changed_when`/`creates` guards, or genuinely non-idempotent logic.
idempotency_score = (unchanged_tasks / total_tasks) × 100
Run the playbook twice in a row against the same infrastructure. The second run's changed-task count (ideally zero) is what you plug into this calculator — a well-behaved playbook should report 'changed=0' on that second run.
Raw `shell`/`command` tasks without `changed_when` (they report 'changed' every time by default), tasks that always overwrite a timestamped file, and templates whose Jinja2 output isn't stable across runs (e.g. embedding the current date).
Nearly always for configuration management tasks, but some legitimately stateful operations (rotating a secret, appending to a log) are inherently non-idempotent by design — flag those explicitly rather than treating a nonzero score as a bug.
Yes in this model — a failure means the playbook didn't successfully converge to the desired state, which is a distinct problem from 'changed' but still means the run wasn't a clean no-op success.