Last week, we discussed The Role of Data Contracts in Master Data APIs and examined how clear agreements between systems can protect the structure, meaning, and quality of master data as it moves through an organization. Data contracts help teams define what an API is expected to provide, how changes should be managed, and what happens when a producer fails to meet those expectations.

This week’s article begins Part 6 of the Master Data Mondays series: Cleanup, Change, and Culture. It also marks the beginning of the final part of the series. Over the coming weeks, we will move beyond identifying master data problems and focus on the practical work of correcting them, managing the organizational changes that follow, and building a culture that supports better data over time. We begin with a practical look at how to scope, prioritize, validate, and control a master data cleanup project.

How to Run a Master Data Cleanup Project

The fastest way to sink a master data cleanup is to start scrubbing before you’ve decided what actually matters, who’s accountable for the outcome, and how you’ll know if it worked.

That’s how these projects turn into bottomless backlogs.

Picture this: a team flags 300,000 possible duplicate customer records and hands them off to data stewards for review. Finance pushes back on the proposed matches. Sales won’t touch anything that might affect account ownership. And while all that’s being sorted out, the CRM keeps churning out new duplicates faster than anyone can clean up the old ones.

The tools aren’t the problem here.

The problem is that nobody actually defined the project.

A real master data cleanup isn’t a one-time scrub-down. It’s a controlled change effort, one that fixes existing defects, shields downstream systems from harm, and makes it less likely the same mess comes back.

The easiest way to manage that kind of effort is to break it into cleanup waves.

Each wave sticks to a limited set of records, rules, systems, and business outcomes. That’s what keeps the work measurable, reviewable, and, if something goes wrong, reversible.

Why Master Data Cleanup Projects Fail

There’s a pretty predictable pattern behind most failed cleanup efforts.

The scope balloons. Nobody owns it. Business teams get looped in too late. Matching rules get treated as a purely technical decision. Thousands of borderline records get dumped into manual review. Changes ship without real testing. And the source systems that caused the mess in the first place never get touched.

A lot of cleanups kick off with something as vague as:

Clean up the customer master.

That’s not a project. That’s a wish.

Which customers, exactly? Which defects? Which systems? What does “clean” even mean here? Who gets to decide two records are the same customer, and what happens when that call turns out to be wrong?

Until someone answers those questions, there’s no cleanup project to speak of. Just a pile of data problems waiting for direction.

A better way to work through it is six stages:

1. Scope
2. Baseline
3. Prioritize
4. Remediate
5. Validate
6. Sustain

Put together, these form a repeatable cleanup wave you can run again and again.

Step 1: Scope One Cleanup Wave

Don’t try to tackle the whole customer, supplier, product, or employee domain at once.

Pick one wave: small enough to keep under control, but big enough that finishing it actually means something to the business.

A first wave might look something like:

• Active commercial customers in North America
• Records created or updated in the past three years
• Duplicate legal entities and incomplete tax identifiers
• Data feeding CRM, billing, and financial reporting
• Accounts tied to active contracts or open invoices

That kind of boundary gives the team something they can actually evaluate.

Every wave deserves a short charter spelling out:

• The master data domain
• Which records are in scope, and which aren’t
• The business problem being solved
• The systems involved
• The defects being targeted
• The business owner
• The data steward
• The technical owner
• How success will be measured
• The release method
• The rollback method

That charter should also tie back to one to three business outcomes, nothing more.

Think fewer returned invoices, better customer matching in analytics, less duplicate account creation, fewer shipments to bad addresses, or cleaner regulatory reporting.

Resist the urge to cram ten or fifteen goals into the charter. A single wave should prove a small handful of things, and prove them well.

Step 2: Profile and Baseline the Data

Once the scope is locked in, profile the data before you touch it.

Profiling tells you what’s actually broken, and it gives you a baseline you can compare against once remediation is done.

At minimum, look at these six dimensions:

Completeness

Are required fields missing?

For customer data, that might mean legal name, billing address, tax identifier, country, account status, or enterprise customer ID.

Validity

Do the values follow the expected format, domain, or business rule?

Think invalid country codes, malformed email addresses, dates that couldn’t possibly be real, or product codes that don’t match approved patterns.

Consistency

Do different systems describe the same thing the same way?

One system might call a customer “Acme Corporation,” another “ACME Corp,” and a third “Acme Holdings.” None of those is technically wrong. They just don’t agree with each other.

Uniqueness

Are multiple records describing the same real-world entity?

Duplicate detection needs to look beyond exact name matches, since standardized matches and probable matches matter too.

Referential integrity

Do related records point to master records that actually exist?

Orders, invoices, service records, contracts, or product assignments can end up referencing master IDs that are gone, or were never valid to begin with.

Timeliness

Is data getting created, updated, and distributed on schedule?

A record can be perfectly accurate and still fail the business if it shows up in a downstream system three days too late.

The baseline should also point to where the defects are actually coming from.

Knowing you have a 12 percent duplicate rate is useful. Knowing that 80 percent of those duplicates trace back to one CRM integration is far more useful.

Here’s a fictional example of what that baseline might look like:

Metric	Starting Condition
Active customer records	240,000
Suspected duplicate records	28,400
Records missing tax identifiers	16,700
Invalid country codes	3,900
Orphaned billing relationships	1,240
Records updated outside the expected service level	21%

None of these numbers tell you what to fix first. They just mark where you’re starting from.

Step 3: Prioritize Defects by Impact, Risk, and Effort

The biggest defect category isn’t automatically the most important one.

A missing phone number might touch 40,000 records and barely register operationally. An incorrect tax identifier might touch only 600, and still create real financial or regulatory exposure.

So the backlog needs to be ranked on more than raw volume.

One workable scoring model:

Priority score = weighted business value and risk, divided by effort

The weighted side of that can factor in:

• Business impact
• Operational or regulatory risk
• Defect volume
• How likely the defect is to come back
• How many systems and processes consume the data

Score each category from one to five.

For example:

Defect	Impact	Risk	Volume	Recurrence	Sharing Breadth	Effort	Priority
Duplicate legal customers	5	5	4	5	5	3	High
Missing phone numbers	2	1	5	3	2	2	Medium
Invalid tax identifiers	5	5	2	4	4	2	High
Inconsistent abbreviations	2	1	5	5	3	1	Medium

The formula won’t replace judgment. It just makes the judgment visible to everyone in the room.

Every score should come with a one-line explanation, or teams will end up arguing about the numbers instead of fixing the defects.

The whole point is a shared, defensible view of what gets tackled first, and why.

Step 4: Clean Data in the Right Order

Work from the safest changes toward the most uncertain ones, not the other way around.

A remediation order that tends to hold up:

1. Deterministic standardization

These are the low-ambiguity fixes.

Trimming spaces, converting country names to approved codes, standardizing capitalization, stripping invalid characters, applying approved date formats: all fair game to automate right away.

2. Validation and rejection

Not every defect should be auto-corrected.

A tax identifier with the wrong number of digits probably needs to be kicked back to the source system or the business owner, not silently “fixed.”

Draw a clear line between values you can correct and values that genuinely need new information.

3. Safe automated fixes

Some defects can be resolved once there’s a trusted source or an approved rule behind the fix.

A missing state code, for instance, might be safely filled in from a verified postal reference table, but only when the city and postal code line up as an exact match.

4. Duplicate candidate generation

Duplicate detection should produce candidates first, not merges.

Potential matches can be pulled from names, addresses, tax identifiers, email addresses, phone numbers, external identifiers, and relationship data.

The matching logic should output a confidence score and show its work: why it thinks these records look alike.

5. Manual review

Save human review for the genuinely uncertain cases.

If everything ends up needing manual eyes, that’s a sign the matching rules aren’t mature yet. Otherwise you’re just building a steward backlog that grows faster than anyone can work through it.

6. Source correction

The cleanup isn’t finished until the source of the defect gets fixed too.

That could mean changes to an application, an integration, a business process, a reference table, a validation rule, a user interface, or who owns what.

Clean up 20,000 duplicates without fixing what created them, and you’ve just reset the clock.

Step 5: Validate Matches, Merges, and Downstream Results

This is where the project proves the data actually got better, not just different.

A solid validation process works across four levels.

Validate the rule logic

Test cleansing and matching rules in a controlled environment.

Throw in normal records, edge cases, incomplete records, conflicting values, international formats, and records that look alike but aren’t the same entity at all.

Rules that sail through easy records can fall apart the moment things get messy.

Validate duplicate matching

Use a labeled sample of record pairs that real business users have already reviewed.

Track:

• Precision
• Recall
• False merges
• False splits
• Manual review volume

Precision tells you how often the proposed matches are right. Recall tells you how many of the real duplicates actually got caught.

False merges are usually the riskier problem. Merge two separate customers by mistake, and you can wreck financial reporting, account ownership, legal agreements, privacy, and service history in one move.

High-confidence matches can eventually be trusted to merge automatically. Medium-confidence matches should go to stewards. Low-confidence matches should stay separate until better evidence shows up.

Validate downstream systems

A technically clean merge can still break something downstream.

Check how the cleanup ripples through:

• CRM accounts and opportunities
• Billing and payment history
• Contracts
• Orders and shipments
• Reporting
• Analytics
• Integrations
• Access controls
• Regulatory records

Make sure identifiers update correctly and that historical relationships don’t get lost in the shuffle.

Validate business outcomes

Circle back to the outcomes you wrote into the charter.

Did returned invoices actually drop? Did customer matching in analytics improve? Did the duplicate creation rate fall? Are reports needing fewer manual fixes?

A cleanup wave isn’t a success just because a script ran without errors.

Step 6: Release Safely and Stop Defects From Returning

Big master data changes need to be reversible. Full stop.

Every record that gets updated, standardized, split, or merged should carry an audit trail with:

• The original value
• The new value
• The rule applied
• The rule version
• The cleanup batch
• The date and time
• Who or what approved the change
• The reason behind the decision

Merge decisions also need to preserve the link between the surviving record and the one that got retired.

Downstream systems may keep sending old IDs for months afterward. You need a reliable cross-reference so those transactions can still be traced.

Before you release anything, actually test the rollback.

Don’t assume backups alone will save you. The team needs to know, ahead of time, how to reverse a cleanup batch, restore retired records, repair cross-references, and replay downstream updates.

Once it’s live, keep watching both quality and recurrence.

Worth tracking:

• Duplicate creation rate
• Required-field completeness
• Invalid-value rate
• Manual review backlog
• False-merge incidents
• Records updated within the expected service level
• Defects by source system
• Defects by business process

The most telling metric might just be how many defects show up after the cleanup. That number tells you whether you fixed the actual process or just tidied up the history.

Master Data Cleanup Project Checklist

Before kicking off a wave, make sure the team can answer these:

• What master data domain is included?
• Which records are in scope?
• Which business outcomes are expected?
• Who owns the data?
• Who approves ambiguous changes?
• Which defects are being targeted?
• What’s the current quality baseline?
• How will defects be prioritized?
• Which corrections can be automated safely?
• Which records need manual review?
• How will matching accuracy be tested?
• Which downstream systems could be affected?
• How will changes be audited?
• How will the cleanup be reversed?
• What source-side controls will prevent recurrence?
• Which measures get monitored after release?

If you can’t answer most of these, you’re not ready to start.

Cleanup Is a Controlled Change Project

A master data cleanup doesn’t succeed just because the database has fewer duplicates in it.

It succeeds when the organization can say what changed, why it changed, who signed off, how it affected the business, and what’s stopping the same defects from coming back.

So start small.

Pick one wave. Build the baseline. Rank the work by impact and risk. Apply the safe fixes first. Send only the genuinely uncertain cases to stewards. Test the changes across every downstream system that touches the data. Keep a complete audit trail. Then go fix the process that caused the mess in the first place.

That path can feel slower than just launching one massive cleanup job.

In practice, it’s usually a lot faster than repairing a bad merge, rebuilding trust in the master data, or cleaning the exact same records again next year.