The DataOpsQuality Manifesto
After years of working with data teams of every size and stack, we’ve come to believe that quality comes only from testing, and that the quality worth testing is the whole delivery, not just the data. Perfect data through the wrong code, report, or model is still an error, and your customer can’t tell the difference. We’ve been the data quality nag. We’ve eyeballed row counts and hoped. We’ve run the 12-month assessment that produced a document and fixed nothing. We’ve built the dashboard nobody opened, and we’ve heard about bad data from the customer at 9 am. We want a method that proves quality at every point, with tests that run without a hero at the keyboard, and evidence we can hand to whoever can fix it. The DataOps Manifesto says quality is paramount. The Data Journey Manifesto says customers finding problems is not OK. This is how we put both to work.
Sign the ManifestoThrough our work, we have come to value:
- Automated tests over hope and manual checks
- Continuous practice over one-time projects
- Measurement over opinion
- Generated tests over hand-written rules
- Test coverage over lineage diagrams
- Quality stations on the line over inspection at the end
- Influence and evidence over mandates
- Working at 70% today over perfect someday
24 principles in six groups, plus one. Select one to jump to it.
1-4
Evidence
-
1
You Only Know What You Tested
Hope is not a test. Neither is a business review or a green pipeline run. If no test checked it, you don’t know it’s right.
-
2
Test The Data, Watch The Tools
Quality is data quality plus process quality. A perfect table is useless if the refresh never ran, and a green job says nothing about the numbers inside. Check every tool for errors and for timing.
-
3
No Manual Checks
Manual checks run when someone remembers, and nobody remembers at 2 am. Your data arrives every day, so every test runs automatically, on every load. The Data Journey Manifesto puts it plainly: avoid manual quality testing like the plague.
-
4
Measure Before You Set Standards
Profile your data, create tests, and let the baseline tell you what the standards should be. A committee writing them first delays every decision after. Without test results, you’re just another person with an opinion.
5-6
People
-
5
Start With One Person
One motivated person with a free tool beats a steering committee with a charter. Find the nag and hand them evidence.
-
6
Influence Is The Job
The person who finds a defect rarely owns the system that made it. Source data is good enough for the job it was built for, so its owner has no reason to change it. Make the fix easy to say yes to.
7-9
Tests
-
7
Let The Profile Write The Tests
Hand-coding hundreds of tests per table never gets finished. Generate them, and save your SQL for the business rules only your domain experts know.
-
8
Tests Are A Shared Resource
The analyst, the steward, the engineer, and the model all touch the same rule, and only one of them has a Git account. Keep every test and every result in one database of record, reachable by a person through a UI, by code through an API, and by a model through MCP. One copy is authoritative, and every change goes to Git for diff, review, and rollback.
-
9
70% Today Beats Perfect Someday
Ship a working test this afternoon. Each loop teaches you something about your data, and the standards improve with every one.
10-13
Scores And Action
-
10
Score What Matters
Nobody changes their behavior over a fax number column. Score the elements your customer cares about, and build a different score for each customer.
-
11
Every Score Traces To A Test
A score without a fix path is noise. Every dashboard number traces to a test that shows the problem and proves the fix.
-
12
Make Acting Easy
Every dashboard has a named customer and a named person who can fix the data. Hand them the problem, its impact, how to reproduce it, and what to change, in the ticket system they already use.
-
13
Keep The Trend Line
One score is an opinion. Six months of scores tied to a KPI someone above you already watches is a budget. Start with two numbers: errors in production and time to ship a change. The trend is what you point at when you ask for more room.
14-19
Practice
-
14
Love Your Errors
Each defect teaches you something about your data or process. Review them together in a quality circle and fix the step that produced them. Blame the step, not the person.
-
15
Put Delivery Terms In Tests
A data contract is a set of tests both sides agree to. Terms you can run beat promises you can’t.
-
16
Prove The Migration
A migration promises that the new system holds the same truth as the old one. Test both sides and prove it, from row counts to column values.
-
17
Cut The Noise
An alert nobody trusts is worse than no alert. Tune every test until a failure means something, and send it to the person who can act on it. Demote the ones that fire every night. Never mute them.
-
18
You Own It In Production
Deploying it is the middle of the job, not the end. Somebody watches the line, and that somebody is you.
-
19
Test What The Agent Wrote
An agent ships a transformation in seconds and keeps none of the pipeline in its head. Let it draft the fix ticket and the custom SQL. Everything else it touches is untested until a test says otherwise.
20-24
Prove Quality At Every Point
-
20
Tests… At The Source
Prove the source is fit for purpose, one table and one column at a time. You don’t own it, so your evidence has to persuade the person who does.
-
21
Tests… At Ingestion
Watch every table for freshness, volume, schema changes, and drift. You can’t watch 6,000 tables by hand. Anomaly detection can.
-
22
Tests… In Production
Put a tripwire between every layer and stop the line on a defect. Not every failure earns a stop, so decide in advance which ones halt and which ones warn. Production stays on the last known good data while someone looks.
-
23
Tests… In Development
Run today’s code against a clone or slice of yesterday’s production data before you deploy. Rename a column, and a red light should tell you which downstream report you broke. A bug caught here costs a hundredth of what your customer finds.
-
24
Tests… Everywhere, Every Time, and Automated
One null check does four jobs in four places. Skip one, and you’ll miss that failure. Cover every table, every column, every tool, and every number your customer reads. No manual tests.
Start DataOps With Quality
The DataOps Manifesto has 18 principles, and nobody adopts them all on Monday. Start with automated data tests. They’re the cheapest step, and your customer feels them first. Orchestration, CI/CD, and observability all depend on knowing whether the data is right.
Sign the DataOps Quality Manifesto
Add your name if you would rather prove quality with automated tests than hope for it.
Put the manifesto to work with open source
DataOps TestGen
Profile your data and generate the quality tests automatically. Full coverage in minutes, with no hand-written rules to maintain.
Install TestGenDataOps Process Observability
Watch every tool and every run from source to the dashboards that depend on it. Catch errors, late arrivals, and bottlenecks before your customer does.
Get Process ObservabilityFurther reading
Free Books
Free Training & Certification
Read the other manifestos
The DataOps Manifesto
18 principles for delivering analytics, plus one: start with data testing.
Read the DataOps ManifestoThe Data Journey Manifesto
22 principles for observing the paths your data takes, so you find the problem before your customer does.
Read the Data Journey Manifesto