DORA Metrics Are Not KPIs

When we first wired DORA into the delivery dashboard, I thought we'd finally settle the argument. Deployment frequency, lead time, change failure rate, recovery time — objective numbers, same definitions, no more debating whether the team was "healthy." Then Q3 OKRs named deployment frequency, and the chart went green before anything about how we shipped changed.

That's the lesson most teams learn in reverse. DORA metrics are useful. They are not KPIs. The moment leadership turns them into targets, Goodhart's law kicks in and the numbers stop measuring what you think they measure.

DORA in One Honest Paragraph

DORA's software delivery metrics split into throughput and instability. Throughput covers change lead time, deployment frequency, and failed deployment recovery time — how fast changes move and how fast you recover when a deploy goes wrong. Instability covers change fail rate and deployment rework rate — how often production needs immediate intervention after a push.

The research is clear that these outcomes correlate with organizational performance and team well-being when teams use them to assess and improve. Top performers tend to do well across the whole set; struggling teams tend to struggle across it. Speed and stability are not enemies in the data — they're usually moving together because the same system properties produce both: small batches, good testing, shared ownership.

That only works if you treat the metrics as diagnostics for one application or service, in context, over time. Not as a league table.

The Scoreboard Trap

Call it dashboard theatre: a line trending up and to the right while the pipeline underneath stays the same. Leadership gets a screenshot; engineers learn which numbers are safe to optimize. The metric becomes the deliverable.

DORA's guide lists the first pitfall plainly — setting metrics as a goal increases the likelihood teams will game them. Here is what that looks like in practice.

Redefining the deploy. Deployment frequency is low, so the definition expands. A config tweak counts. Re-running the pipeline counts. Splitting a one-line README change into its own release counts. Suddenly you're "elite" on frequency while still batching real features into monthly drops.

Shrinking the failure. Change failure rate is embarrassing, so the bar for "failure" rises. A rollback isn't a failure if users didn't notice. A hotfix isn't a failure if it shipped within the hour. The rate falls. Incidents don't.

Hiding work upstream. Lead time improves when measurement starts later. Commit-to-deploy looks fast because code review and QA wait in someone else's clock. The DORA number is green; the calendar still shows three weeks.

Optimizing the chart, not the pipe. Someone gets assigned to make the dashboard presentable. Integrations ship. Tier badges appear. Nobody can answer what changed in the delivery process because nothing did — except what gets counted.

I've sat in the readout where a VP congratulates a squad on moving from "Medium" to "High" tier while the same three engineers still manually babysit the release checklist every Thursday. The tier moved. The checklist didn't. That's not improvement — it's classification theatre with a vendor logo on it.

These four moves are the same pattern wearing different masks: when a measure becomes a target, it stops being a good measure. A dashboard with an executive audience and no curiosity behind it is a target factory.

What DORA Already Told You Not to Do

This isn't a hot take from someone who hates metrics. The official guidance reads like a warning label:

• Don't set metrics as goals. Goodhart isn't a footnote — it's listed under common pitfalls.
• Don't compete across teams. Improve against your own baseline, not someone else's mobile app vs your mainframe batch job.
• Don't worship one number. Use a balanced set with healthy tension between them.
• Don't roll up disparate contexts. Metrics fit one service; org-wide blends erase the context that makes them meaningful.
• Don't silo ownership. Splitting throughput metrics for dev and stability metrics for ops recreates the wall the research argues against.

The default enterprise rollout anyway: buy a tool, paint elite-tier benchmarks on the wall, tie quarterly objectives to deployment frequency, compare squads in a staff meeting. Every step violates the manual.

The 2024 Accelerate report adds a sharper edge: more AI in the development process didn't automatically improve delivery outcomes when teams skipped the basics — small batches, robust testing. Measuring harder is not a substitute for changing how work moves. You can adopt copilots and still ship the same monolithic PRs on the same brittle pipeline. The dashboard won't catch that until the failure modes surface somewhere else.

Leading Indicators Before Lagging Scores

DORA keys are lagging indicators. They tell you what already happened at the end of the pipe. Useful — but late for weekly steering.

MinimumCD's anti-pattern writeup puts the alternative cleanly: track behaviors that cause delivery performance to improve, against your own past, for your own deployment context. Integration frequency trending up as branches shorten. Stories finishing in days instead of weeks. Broken builds treated as stop-the-line events with sub-ten-minute fixes. Two to four improvement experiments per month, each tied to a hypothesis about a bottleneck.

Picture a team whose deployment frequency flatlined for a year. The lagging chart won't move until someone shortens review wait — say from four days to same-day on small PRs — and keeps batch size honest. DORA's recommended loop matches that spirit without requiring a six-month tooling project: baseline with the Quick Check, map friction in the delivery path, commit to one constraint, define leading measures (review wait time, test reliability, batch size), do the work, re-check. Lagging DORA metrics move when the system moves — often on a 3–6 month horizon, not the next sprint review.

Pair speed with stability so gaming one shows up in the other. Deployment frequency rising while change fail rate spikes tells you something. Both flat while leading indicators climb tells you something else — and that second something is what teams who want health, not a badge, should be watching.

The Question That Separates Practice from Theatre

Practitioner Francesco Latini frames the test in one sentence: What did this number cause us to change?

In theatre, the number is the work — green cell, quarterly deck, done. In practice, the number is a question. Bad lead time sends you to the slowest stage. A recovery time that only measures internal dashboards, not user impact, gets re-instrumented even if the number gets worse before it gets honest.

If your team can name the bottleneck they killed because a metric moved, you're doing metrics. If the only artifact is a screenshot, you're doing KPI theatre with DORA vocabulary.

DORA metrics belong on the retrospective whiteboard, not the performance scoreboard. Track trends for the service you own. Keep the balanced set. Ask what changed — not who beat elite tier.

The chart is a thermometer. You still have to treat the patient.