Data Quality Part 2: Fixing It - Critical Data Elements, Contracts, and the One Question That Stops Robodebts

Part 2 of 2 in our Data Quality series.

In Part 1, James came in skeptical and walked out sold on the problem. In Part 2, we deliver the fix — the discipline, the architecture, and the eight concrete moves executives can make on Monday morning. This is the episode for leaders who heard last week's case studies and asked "okay, but what do we actually do?"

What we cover:

• The one question every CEO should be asking this week: what are our Critical Data Elements, who owns each one, and how do we know each is fit for purpose?
• Why fixing all the data is how data quality programs die — and how ruthless tiering (50-300 fields, not 50,000) is how they survive
• Data contracts: the quiet revolution in how serious organisations manage producer-consumer relationships, popularised by Andrew Jones at GoCardless and Chad Sanderson
• The five default checks every Critical Data Element should pass: freshness, volume, schema, distribution, referential integrity
• The five-layer reference architecture: contracts, validation, observability, lineage, governance — and why governance is where most organisations fail
• Unity Technologies 2022: how contaminated training data cost $110M in revenue and $5B in market capitalisation in a single day
• Robodebt: the Australian government program that issued ~470,000 invalid debt notices, ended in a Royal Commission, and cost $1.8B in settlement — and the three-word question that would have stopped it
• The eight-step Monday-morning move: a complete executive action plan
• The case study James can't name: a global enterprise (90,000 people, $50B+ revenue) six years into a serious data strategy — with every right concept on paper, an aggressive AI rollout underway, and a green dashboard hiding the reality. Why "the mandate is not the implementation" is the most dangerous gap in enterprise AI today.

The one question that stops Robodebts: "Fit for purpose for what?"

Key references:

• Wang & Strong (1996), foundational dimensions of data quality: https://doi.org/10.1080/07421222.1996.11518099
• DAMA UK — Six Core Data Quality Dimensions: https://www.sbctc.edu/resources/documents/colleges-staff/commissions-councils/dgc/data-quality-deminsions.pdf
• Critical Data Elements Explained: https://www.dataversity.net/articles/critical-data-elements-explained/
• ISO/IEC 25012:2008 — Data Quality Model: https://www.iso.org/standard/35736.html
• Sambasivan et al., "Everyone wants to do the model work, not the data work" — data cascades in high-stakes AI (Google Research, CHI 2021): https://research.google/pubs/everyone-wants-to-do-the-model-work-not-the-data-work-data-cascades-in-high-stakes-ai/
• IBM Institute for Business Value — 2025 CDO Study: https://www.ibm.com/thought-leadership/institute-business-value/en-us/report/2025-cdo
• BCBS 239 — Principles for effective risk data aggregation and risk reporting: https://www.bis.org/publ/bcbs239.htm
• Royal Commission into the Robodebt Scheme — Final Report (2023): https://robodebt.royalcommission.gov.au/publications/report
• Unity Technologies Data Quality Issue: https://www.fool.com/investing/2022/07/17/2-reasons-unity-softwares-virtual-world-is-facing/
• Andrew Jones — Driving Data Quality with Data Contracts: https://andrew-jones.com/data-contracts-101.pdf
• Chad Sanderson — The Rise of Data Contracts: https://dataproducts.substack.com/p/the-rise-of-data-contracts
• Chad Sanderson — Data Products and Contracts (Data Quality Camp): https://www.youtube.com/watch?v=1CSTSdfe0qg

If this series helped, share it with the loudest voice on AI strategy in your organisation. If their AI strategy doesn't have a data quality strategy underneath it, you now know what to ask them.

Better AI still starts with better foundations.

Transcript

SPEAKER_01 0:08

Okay, straight into it today. Sarah, last episode I came in as the skeptic. I said data quality felt over-engineered. By the end of it, I was sold on the problem. PHE losing 15,000 cases in a spreadsheet. NASA losing $327 million and a spacecraft because two teams couldn't agree on units. City wiring $894 million by accident. Today, I want the fix.

SPEAKER_00 0:38

You also closed last episode by making me audit this podcast against its own seven dimensions. Live. On air.

SPEAKER_01 0:46

And you gave us six out of seven, pending part two for completeness.

SPEAKER_00 0:50

Which is the whole reason we're here. So let's earn that seventh point. Today is the answer to what do I actually do on Monday? Not vibes, not self-assessment, real measurement, real architecture, real decisions.

SPEAKER_01 1:08

With real consequences when you get it wrong.

SPEAKER_00 1:10

Which is where we're going to land at the end of this episode, with the most consequential data quality failure of the modern era. 470,000 people. $1.8 billion. A Royal Commission.

SPEAKER_01 1:26

Robodebt.

SPEAKER_00 1:27

Robod. But before we get there, we need to give executives the tools to make sure they never end up there.

SPEAKER_01 1:33

And Sarah, before we start, I want to keep one organization in mind as we go through this. I'm not going to name them. But I've spent time with a company, one of the largest in its global industry, around 90,000 people, north of $50 billion in annual revenue, that's been on a serious data strategy journey for over six years.

SPEAKER_00 1:54

Okay.

SPEAKER_01 1:55

They've done a lot of the things we're about to describe: owners, stewards, critical data elements, federated structure, company-wide standards published and signed off. And they are right now rolling out an aggressive AI strategy on top of all of that.

SPEAKER_00 2:11

And how's it going?

SPEAKER_01 2:12

That's the question. I want to come back to them as we go through each of the mechanisms today. Because they're a really useful mirror. They show what happens when you put the right concepts in place, but miss the conditions that make them actually work.

SPEAKER_00 2:26

Polished on the surface.

SPEAKER_01 2:28

Don't. Sorry. Welcome to AI Beyond the Hype.

SPEAKER_00 2:33

I'm James. And I'm Sarah. Part two of two on data quality. Today we fix it.

SPEAKER_01 2:40

Okay, Sarah, give me the one question. If a CEO listening to part one is now genuinely worried, what's the single question they should be asking their leadership team this week?

SPEAKER_00 2:50

One question. Okay, here it is. What are our critical data elements? Who owns each one? And how do we know each one is fit for purpose right now?

SPEAKER_01 3:01

That's three questions.

SPEAKER_00 3:03

It's three questions that fit on one slide, and most organizations can't answer any of them.

SPEAKER_01 3:09

Walk me through it. Critical data elements. What does that actually mean?

SPEAKER_00 3:13

So, and this is the bit I get weirdly excited about. Not all data matters equally. Most organizations have tens of thousands of data fields, and the instinct when you start a data quality program is to try to fix all of it. That's how programs die. Two years in, no measurable progress, the budget gets cut, everyone declares data quality a failed initiative.

SPEAKER_01 3:39

I've seen that movie.

SPEAKER_00 3:40

Everyone has. The fix is to be ruthlessly selective. A critical data element, a C D E is a piece of data where, if it's wrong, something material happens, a customer makes a wrong decision. A regulator find you, a safety incident occurs, revenue is misstated. Most organizations, when they sit down and do this exercise honestly, find their truly critical data elements are somewhere between 50 and 300 fields.

SPEAKER_01 4:17

Out of tens of thousands.

SPEAKER_00 4:19

Out of tens of thousands. And here's the line I want executives to hold on to.

SPEAKER_01 4:31

That's the bit my CFO would actually believe. Because nobody on an exec team buys, we're going to fix all the data. They might buy, we're going to fix the 200 fields that move the PL and the risk register.

SPEAKER_00 4:44

And that's the difference between a strategy and a slide.

SPEAKER_01 4:48

Don't steal my line. And Sarah, the company I mentioned at the top, they've done this exercise. They've formally identified critical data elements. It's in the standard. The standard says CDEs will be identified and data quality assessments will be completed against them.

SPEAKER_00 5:05

So they've ticked the box.

SPEAKER_01 5:07

On paper, absolutely. In practice, partially. The mandate exists. The execution against it is patchy. And that gap between we have a standard and the standard is actually implemented on the ground is where we're going to spend a lot of this episode.

SPEAKER_00 5:24

That gap is where most organizations actually live, by the way. The standard is the easy part. Execution is the hard part.

SPEAKER_01 5:34

Okay, so how does an organization actually identify these? Because I can imagine 50 different department heads each insisting their data is critical.

SPEAKER_00 5:43

Yes, that's exactly what happens. So you need a forcing function. The approach that works, and this comes out of the regulatory world, particularly BCBS 239 in banking, which requires banks to identify critical data for risk aggregation and reporting, and to apply governance and controls proportionate to that criticality. Generalizing this concept, we can think of tiering your data based on consequence of failure.

SPEAKER_01 6:12

Tier 1, 2, 3.

SPEAKER_00 6:15

Tier 1. If this data is wrong, there is a material consequence within 24 hours. Regulatory breach, safety incident, customer financial harm, board reportable event. Tier 2. If this data is wrong, there's a meaningful operational or financial consequence within a week. Tier 3. Everything else. Useful, valuable, but not critical.

SPEAKER_01 6:45

And the discipline is Tier 1 gets the heavy controls.

SPEAKER_00 6:48

Tier 1 gets the contracts, the monitoring, the alerts, the named owner, the SLA. Tier 2 gets lighter controls. Tier 3 gets best effort. That's it. That's how you stop a data quality program from collapsing under its own weight.

SPEAKER_01 7:08

Sarah, I want to push back. Because Tier 1 in financial services is reasonably obvious: capital adequacy, anti-money laundering, customer onboarding. What does Tier 1 look like in a less regulated business? Like a retailer or a manufacturer?

SPEAKER_00 7:23

Great question. In a retailer, pricing data, inventory data, customer contact for safety recalls. If those are wrong, you have a customer harm event, a margin event, or a recall event. In a manufacturer, and this connects to your story from last episode, safety inspection data, asset condition data, hazard zone definitions. If those are wrong, you have latent risk in your operation.

SPEAKER_01 7:51

Right, and that's the example I keep coming back to. Because at the time, nobody on that site would have called that data tier one. It was just inspection paperwork. But the consequence of it being wrong was, eventually, that the operational picture didn't match reality.

SPEAKER_00 8:08

Latent risk. And that's exactly the kind of thing this exercise catches. When you ask, if this data is wrong, what happens? And you ask it honestly, the safety-relevant data leaps to the top of the list. It just doesn't always live in the places executives are looking.

SPEAKER_01 8:27

Okay, so we've got our 200 critical data elements. Now what?

SPEAKER_00 8:32

Now we make them contractual. And this is the idea that's quietly reshaping how serious organizations are managing data. The concept is called a data contract.

SPEAKER_01 8:43

Sounds legalistic.

SPEAKER_00 8:44

It sounds legalistic on purpose. The idea, and the people who've done the most to define this, are Andrew Jones at GoCardless and Chad Sanderson, who's written extensively about it. The idea is that the team producing the data signs a contract with the teams consuming it. Schema, freshness, completeness, validity rules, all explicit, all versioned, all enforced.

SPEAKER_01 9:09

Like an API contract, but for data.

SPEAKER_00 9:12

Exactly that. And it sounds boring. And it is boring. And that's the point. Because for 30 years, the way enterprise data has moved between systems is engineering team A builds a pipeline, engineering team B uses the output, and when A changes something upstream, B finds out by the dashboard breaking.

SPEAKER_01 9:34

Or by the regulator calling.

SPEAKER_00 9:36

Or by the regulator calling. And the data contract approach says, no. If team A wants to change the schema, they propose the change, the consumers review it, they agree on a migration path. The data is treated as a product with a defined interface.

SPEAKER_01 9:53

Sarah, what does this look like for executives? Because I can already feel listeners thinking this is engineering stuff, not my problem.

SPEAKER_00 10:00

Three things matter at the exec level. First, somebody has to own the data product. A named person, not a committee, not the platform team. Owners with names. Second, the SLA has to be agreed cross-functionally. Finance can't unilaterally declare what good enough means for marketing's data. Third, there has to be a budget for the producer team to invest in the quality of what they're publishing. That last one is where most organizations fail. They expect data quality to come for free, alongside whatever the producer team's actual KPIs are.

SPEAKER_01 10:39

And it doesn't come for free.

SPEAKER_00 10:41

It never comes for free. And if you don't fund it, you get what you've always gotten. Polished on the surface, shaky underneath.

SPEAKER_01 10:49

I'm allowed to say that, you're not. Noted. Sarah, this is exactly where my mystery company has its sharpest pain. They have data owners, they have stewards. The roles are defined. The names are even in the company-wide data catalogue.

SPEAKER_00 11:05

Let me guess. The owners and stewards have day jobs.

SPEAKER_01 11:08

The owners and stewards have day jobs, and data ownership wasn't initially worked into the job descriptions, wasn't worked into the performance objectives, wasn't worked into the time allocation. So you have these very capable people who have been told, you own this critical data, and are also being measured against their actual operational delivery, and there are only so many hours in a week. And so the ownership becomes nominal. Nominal. The org chart says it's owned. The reality is that owner is firefighting their day job, and the data ownership work happens when it can.

SPEAKER_00 11:44

And then on the producer side.

SPEAKER_01 11:47

No budget. No funded mandate to invest in the quality of what they're publishing. The consuming teams have all the budget. Because they're the ones with the business case, they're the ones with the AI use case, they're the ones with the executive sponsor. So when it comes time to put data quality rules around a CDE, the consuming team makes the call and they skip it. Because their incentive is to get their hands on the data quickly, not to slow down and harden the source.

SPEAKER_00 12:16

And that is exactly the failure mode I was describing. The producer team is unfunded. The consumer team is incentivized against quality. The mandate exists in the standard. Nobody on the ground has the time, the budget, or the incentive to actually do it.

SPEAKER_01 12:34

And the wheels look like they're still on because the data is flowing and the dashboards are green. Okay, so we've identified our CDEs, we've put contracts around them. Now, what are we actually checking? Because when I imagine my data team coming back to me, I don't want a hundred-page assurance document. I want to know what they're actually testing.

SPEAKER_00 12:56

Right, so this is the practical layer underneath everything we've talked about. Five default checks. Every critical data element gets these at minimum. Go. 1. Freshness. Is the data arriving when it should? If the daily customer feed normally lands at 6 a.m. and today it's 10am and nobody noticed, that's a failure. 2. Volume. Is the amount of data within expected range? If you normally get 100,000 records and today you got 800, something is wrong, even if every one of those 800 is technically valid.

SPEAKER_01 13:37

800 clean records is still wrong.

SPEAKER_00 13:40

800 clean records is still wrong. 3. Schema. Are the fields, types, and structures what we agreed in the contract? 4. Distribution. Are the values within expected statistical ranges? If a numeric field that normally averages around 50 suddenly averages around 5,000, that's a flag, even if no individual value violates a rule.

SPEAKER_01 14:09

And 5?

SPEAKER_00 14:09

5. Referential integrity. Do the relationships hold? Every transaction has a real customer. Every order has a real product. Every employee has a real cost center. Five checks.

SPEAKER_01 14:24

That's manageable.

SPEAKER_00 14:25

That's the point. Five checks on a couple of hundred CDEs run continuously. You'll catch the vast majority of consequential data quality failures before they propagate. And you'll catch them at the source, not three systems downstream after they've already produced a wrong report or trained a wrong model.

SPEAKER_01 14:45

Catch them at the source. That's the discipline.

SPEAKER_00 14:48

That's the whole game.

SPEAKER_01 14:50

Sarah, that point about dashboards being green, my company again. They show data quality to executive leadership as a single number.

SPEAKER_00 14:59

One number.

SPEAKER_01 15:00

One number, a roll-up, a traffic light. And the metric underneath it is aligned to something easy to measure, rather than something that reflects fitness for purpose.

SPEAKER_00 15:10

Which is exactly the warning I gave in part one. If your data quality metric is whatever was easiest to measure, you're not measuring data quality. You're measuring measurability.

SPEAKER_01 15:22

And the result is that the indicator is sitting green, has been sitting green for a long time, and the on-the-ground reality is very different. The more mature owners and stewards know it's not green for them. Some of them have built their own bespoke quality tooling because the enterprise solution doesn't actually tell them what they need to know. But that's the exception. It's not the norm.

SPEAKER_00 15:45

So you've got this two-track reality. The enterprise dashboard says everything is fine. The owners who care and who've gone out of their way to build something know it isn't. And the executives looking at the dashboard have no idea there's a delta.

SPEAKER_01 16:01

Until something goes wrong and somebody asks how it could possibly have happened when the indicator was green.

SPEAKER_00 16:07

And this is the most dangerous failure mode of all. Worse than no monitoring. Because no monitoring at least leaves an executive uncertain. False green monitoring leaves an executive confident. And confident executives commit to AI strategies on top of foundations they believe are solid.

SPEAKER_01 16:27

Bad advice delivered with executive presence.

SPEAKER_00 16:30

Now you can use that one. Five layers, top to bottom. And I'll keep it concrete. Go. Layer one contracts. The agreed interface between producer and consumer. Versioned, owned, signed off. Layer two, validation. The checks we just talked about, running at ingestion, before bad data gets into your warehouse or your feature store. Layer three, observability. Continuous monitoring of those checks over time. Trends, anomalies, drift. So you don't just know that today's data is clean, you know how the quality is moving over weeks and months.

SPEAKER_01 17:23

That's a board-level view, potentially.

SPEAKER_00 17:25

It absolutely can be. The chief data officer of a serious organization should be able to walk into a board meeting and say, our tier one CDEs are at 98% quality this quarter. Here are the three that are trending down. Here's why. That's a mature posture.

SPEAKER_01 17:44

Honestly, I couldn't have said that better myself. We may need to reverse roles. Okay, let's move on. Layer 4.

SPEAKER_00 17:52

Layer 4. Lineage. Where did this data come from? Who touched it? What did it pass through? Because when something goes wrong, the first question every executive asks is: where did this number come from? If you can't answer that in 15 minutes, you've got a lineage gap. And layer five, governance. The roles, the policies, the decision making. Who can change a schema? Who signs off on a new CDE? Who's accountable when something breaks?

SPEAKER_01 18:26

And governance is where most organizations actually fail.

SPEAKER_00 18:29

Governance is where most organizations fail. Because the first four layers are technical and viable. You can pick tools, you can hire engineers. Governance is cultural and political. It requires executive sponsorship. It requires saying out loud that certain people own certain data, and that ownership comes with both authority and accountability.

SPEAKER_01 18:52

Which is hard because most organizations have spent 20 years making sure nobody owns anything specific enough to be held accountable for it.

SPEAKER_00 19:01

That is the most depressingly accurate sentence I've heard you say.

SPEAKER_01 19:04

Thank you. I think. Okay, Sarah, before we get to Robodebt, give me one more business case. Because I want one that hits the PL directly. Not government, not safety, just commercial.

SPEAKER_00 19:17

Unity Technologies, 2022. Unity makes the game engine, software used to build a huge proportion of the world's mobile and PC games. They also run an advertising business inside that engine, which is where a lot of their growth was coming from.

SPEAKER_01 19:35

I remember this.

SPEAKER_00 19:36

Their ad targeting model was being trained on customer data, including some data ingested from a major partner. That data had quality issues. Specifically, it had been contaminated by signals from bad actors. Bots, essentially. And Unity's pipeline didn't catch it. The model trained on contaminated data. Targeting accuracy collapsed. Revenue from that business cratered. What was the number? They publicly attributed a $110 million revenue impact to it. But the market reaction was much bigger. The share price dropped roughly 37% in a single day. $5 billion of market capitalization evaporated.

SPEAKER_01 20:22

5 billion?

SPEAKER_00 20:23

Because investors don't just price the immediate revenue hit, they reprice the trust. They ask, if this can happen once, how do I know it's not happening elsewhere? And the answer Unity could give at the time was, we didn't have the controls to detect this at the source, so honestly, we're not sure.

SPEAKER_01 20:44

That's the dangerous bit. Not the 110 million, the 5 billion in confidence.

SPEAKER_00 20:50

And that's the difference between a data quality problem and a data quality crisis. The problem is the bad data. The crisis is when the market realizes you don't have the architecture to know whether your other data is good.

SPEAKER_01 21:04

Sarah, I want to come back to the company I've been threading through this episode. Because for the people actually listening, this case study might matter more than Unity or Robodet.

SPEAKER_00 21:14

Stronger claim than I expected.

SPEAKER_01 21:16

Hear me out. Six plus years of serious investment, one of the largest in their global industry. 90,000 people. Revenue north of 50 billion. They have data owners, stewards, an enterprise standard mandating CDEs and quality assessments, a federated structure preserving regional context. On paper, they've done almost every single thing you and I have recommended.

SPEAKER_00 21:42

And on the ground?

SPEAKER_01 21:44

Owners and stewards without time. CDE work patchy. Quality assessments skipped in many places. No real way for an owner to see the quality of their data. A few of the mature ones built bespoke tooling, but that's the exception. Enterprise dashboards should. Shows a single green number. Producers have no quality budget. Consumers have all the budget and consistently skip quality work because it slows them down. And on top of all of it, an aggressive enterprise AI strategy is rolling out right now.

SPEAKER_00 22:16

James, that's the most common story in enterprise data right now. And almost nobody is telling it honestly. That's why I wanted to bring it. Let me name what's going on, because I want listeners to recognize it if it's their company. Every piece is correct. Structure, roles, standard, federation. All correct. What's missing isn't the design, it's the conditions that turn the design into reality. Owners need time and incentive. Producers need budget. Executives need a metric that reflects truth, not convenience. And the standard needs enforcement, not just publication. And without those conditions, you have an organization that's done the work to look mature without becoming mature. Which is more dangerous than not starting. Because leadership genuinely believes the foundations are solid. They've signed the standard, appointed the owners, seen the green dashboard. So when the AI strategy lands on those foundations, nobody asks the hard question.

SPEAKER_01 23:24

Which is the question from part one. Is the data actually fit for purpose?

SPEAKER_00 23:29

And here the answer is yes, because the dashboard says so. The system is confidently telling you something that isn't true.

SPEAKER_01 23:37

Polished on the surface.

SPEAKER_00 23:38

Shaky underneath. And to be clear, James, these are good people. The owners care. The stewards care. The leadership team is trying. It's not negligence. It's a gap between the design of a system and the conditions that let it function.

SPEAKER_01 23:56

And there's a follow-on series in this, by the way. Because some of what I just described isn't a data quality problem in the pure sense. It's an organizational design problem. How you fund producers, how you structure ownership against day jobs, how a federated, capital project-driven operating model coexists with central data discipline. That's a future conversation.

SPEAKER_00 24:19

Noted. But for now, if you're an executive and parts of that story sounded familiar, please don't interpret we've done the work as the foundations are solid. Those are different claims. Only one of them is what your AI strategy actually depends on.

SPEAKER_01 24:36

The mandate is not the implementation.

SPEAKER_00 24:39

Yep. The mandate is not the implementation.

SPEAKER_01 24:43

Okay, Sarah, Robodet. Walk us through it. And I want listeners, particularly those outside Australia, to understand why this is the case study every executive should know.

SPEAKER_00 24:53

Robodet was an Australian government programme. It ran from 2015 to 2019 under what became known as the Online Compliance Intervention. The premise was: there are people who have received welfare payments and also earned employment income, and we want to detect cases where those two things should have offset and the person was overpaid.

SPEAKER_01 25:18

Reasonable policy goal in the abstract.

SPEAKER_00 25:20

Reasonable policy goal. The problem was in how they did it. The system took annual employment income data from the tax office, divided it by 26 fortnights, and compared that average against fortnightly welfare payments. If there was a mismatch, the system raised a debt notice.

SPEAKER_01 25:39

And the data quality problem is annual income divided by 26 is not the same as actual fortnightly income.

SPEAKER_00 25:47

Exactly. Because real people don't earn evenly across the year. Someone might earn nothing for six months and then work intensely for three. The annual figure is correct. The fortnightly average is meaningless. So the system was systematically generating false debts against people who hadn't actually been overpaid. How many people? They had to prove they didn't owe it. There were documented cases of debt collectors being involved. There were people, including young people in vulnerable circumstances, who took their own lives.

SPEAKER_01 26:35

Hmm. Right.

SPEAKER_00 26:38

A Royal Commission was established. It reported in 2023. The settlement was $1.8 billion. The Royal Commission's findings were scathing about both the technical design and the governance. Specifically, that internal advice flagging the methodology as legally questionable was overridden, and that warnings about the data quality of the income averaging approach were ignored.

SPEAKER_01 27:04

Sarah, the question I want every executive to sit with, and I mean really sit with, is this. At what point in that chain could a serious data quality conversation have stopped this?

SPEAKER_00 27:14

Many points. But if I had to pick one, the very first time someone asked, is this fit for purpose? Annual income divided by 26 used to make legally binding individual decisions about fortnightly entitlements. If anyone with data quality authority had asked that question, fit for purpose for what? The answer would have come back immediately. This data is not fit for this purpose. And that's the question. That's the question. Fit for purpose for what? Three words that, if asked seriously at the right point in the design of that system, would have prevented every one of those 470,000 debt notices.

SPEAKER_01 28:00

The one question that stops Robodebts. Sarah, we've covered a lot. Critical data elements, contracts, five checks, five-layer architecture, Robodebt, Unity. Bring it home. What does an executive actually do on Monday?

SPEAKER_00 28:16

Eight things. In order. And I'll go fast.

SPEAKER_01 28:18

Go.

SPEAKER_00 28:19

One, commission a critical data element inventory. Not all data. The data that, if wrong, matters. Two, assign named owners. Real people. 200 names. And critically, work the ownership into their job descriptions and their performance objectives. Not a side of desk request. Real time, real measurement.

SPEAKER_01 28:47

That one's for my mystery company.

SPEAKER_00 28:49

That one's for everyone, honestly. 3. Agree the tier for each one. Tier 1, tier 2, tier 3. 4. Implement the five default checks on every tier 1 CDE. Freshness, volume, schema, distribution, referential integrity.

SPEAKER_01 29:12

Halfway.

SPEAKER_00 29:13

5. Publish a data quality dashboard for Tier 1, visible at the executive level, quarterly to the board. And the metric has to reflect fitness for purpose, not whatever was easiest to compute. If it's always green, it's probably wrong.

SPEAKER_01 29:29

Make this real for leadership. Better visibility, better decisions.

SPEAKER_00 29:34

6. Write data contracts for the most critical producer-consumer relationships. Start with the top ten. Don't try to do them all at once. 7. Establish a governance forum that can actually decide things. Schema changes. New CDEs. And eight, and this is the most important one, and the one almost every organization skips. Fund the producer teams to invest in quality. Don't expect it for free. Build it into the operating budget. If your producers don't have a budget, your standard is decorative.

SPEAKER_01 30:18

Eight things.

SPEAKER_00 30:19

Eight things. None of them require new technology you don't already have. All of them require executive sponsorship. And all of them are recoverable from where most organizations are today, in 12 to 18 months, if the sponsorship is real.

SPEAKER_01 30:35

Sarah, last episode you audited this podcast against the seven dimensions. Six out of seven. Pending part two for completeness.

SPEAKER_00 30:43

I did.

SPEAKER_01 30:44

Are we awarding the completeness point?

SPEAKER_00 30:47

Let's see. We've now covered the problem and the fix. We've defined what good data is, we've shown what it costs when it's bad, and we've given listeners eight concrete moves to make on Monday. I think, provisionally, we can award completeness.

SPEAKER_01 31:04

Provisionally?

SPEAKER_00 31:05

Provisionally, because completeness is contextual. If a listener walks away thinking, I now know exactly what to do, then yes. If they walk away thinking, I now know what to ask, that's still completeness for an executive-targeted podcast. We're not trying to make them data engineers.

SPEAKER_01 31:26

Which is fortunate because we have 90 minutes, not a four-year degree.

SPEAKER_00 31:30

Seven out of seven. We earned it.

SPEAKER_01 31:32

Excellent. Two episodes. Two big ideas. Episode one. Most good data claims are hope, not measurement. Episode two. There is a real, well-defined, executable discipline for fixing this, and the organizations that take it seriously are the ones whose AI investments actually pay off.

SPEAKER_00 31:52

And the organizations that don't end up somewhere on the spectrum between Unity and Robodebt. Or somewhere on the spectrum between mandate and implementation, which is where most of them actually live.

SPEAKER_01 32:16

The fix is not glamorous. It's owners with names and time, contracts with teeth, producers with budgets, and the discipline to ask fit for purpose for what before you build the thing.

SPEAKER_00 32:28

And a coming series, I think, on the organizational design questions underneath all of this. How you structure ownership, fund quality, and run a federated operating model without losing enterprise discipline. That's the conversation that comes next.

SPEAKER_01 32:44

One to look forward to.

SPEAKER_00 32:45

If you found this series valuable, the most useful thing you can do is share it with the person in your organization who is currently the loudest voice on AI strategy. Because if their AI strategy doesn't have a data quality strategy underneath it, you now know what to ask them.

SPEAKER_01 33:02

Thanks for listening to AI Beyond the Hype. I'm James.

SPEAKER_00 33:05

And I'm Sarah. Next episode, we're back on a new topic. But for anyone who wants to go deeper on what we covered today, full references and links are in the show notes.

SPEAKER_01 33:15

And remember, better AI still starts with better foundations.

SPEAKER_00 33:20

Even when the foundation is unglamorous.

SPEAKER_01 33:22

Especially then.