The data warehouse team scrubs customer names and tax IDs weekly; business enters new bad formats next week. Dirt starts at entry: no format, no dedup, no master-data reference. Post-hoc cleaning pays interest on design debt—interest grows with the business. Frontend validation—format, required, dedup, master-data reference—costs less than expanding cleaning teams and hits the root cause.
Cleaning still matters for history and external data. Incremental should be blocked at the keyboard.
Blocking Beats Washing
Tax IDs, phones, emails, material codes use rules and check digits. New customers force dedup. Addresses split to province/city/district. Free-text remarks must not carry fields that should be structured. When blocked, give actionable hints—not only "format error."
- Imports use the same validation as screens—Excel must not bypass.
- Allowed exceptions need requests—dirty data must have an owner.
- Quality metrics track incremental pass rate—not only historical scrub tonnage.
Cleaning Teams Should Shrink, Not Grow
The XYN digital intelligence system puts master data and format validation at entry and interfaces—analytics eat clean increments. Cleaning teams shift to rule maintenance and exception arbitration—not permanent erasers. Pricing dirt at the source (blocked UX) beats pricing it in reports (bad decisions).
Count how many new customers this week failed format validation. If the rate does not drop, add validation before headcount.
