Data Quality Rules provide the foundation for operational management of Data Quality. Rules can be integrated into application services or Data services that supplement the Data lifecycle, either through Commercial-Off-The-Shelf (COTS) data quality tools, rules engines and reporting tools for monitoring and reporting, or custom-developed applications.
The process of profiling and analyzing data will help an organization discover (or reverse engineer) business and data quality rules. As the data quality practice matures, the capture of such rules should be built into the system development and enhancement process. Defining rules upfront will:
- Set clear expectations for data quality characteristics
- Provide requirements for system edits and controls that prevent data issues from being introduced
- Provide data quality requirements to vendors and other external parties
- Create the foundation for ongoing data quality measurement and reporting
In short, data quality rules and standards are a critical form of Metadata. To be effective, they need to be managed as Metadata. Rules should be:
- Documented Consistently: Establish standards and templates for documenting rules so that they have a consistent format and meaning.
- Defined in Terms of Data Quality Dimensions: Dimensions of quality help people understand what is being measured. Consistent application of dimensions will help with the measurement and issue management processes.
- Tied to Business Impact: While data quality dimensions enable understanding of common problems, they are not a goal in-and-of-themselves. Standards and rules should be connected directly to their impact on organizational success. Measurements that are not tied to business processes should not be taken.
- Backed by Data Analysis: Data Quality Analysts should not guess at rules. Rules should be tested against actual data. In many cases, rules will show that there are issues with the data. But analysis can also show that the rules themselves are not complete.
- Confirmed by SMEs: The goal of the rules is to describe how the data should look. Often, it takes knowledge of organizational processes to confirm that rules correctly describe the data. This knowledge comes when subject matter experts confirm or explain the results of data analysis.
- Accessible to all Data Consumers: All data consumers should have access to documented rules. Such access allows them to better understand the data. It also helps to ensure that the rules are correct and complete. Ensure that consumers have a means to ask questions about and provide feedback on rules.
