Microsoft Purview Data Quality is a cloud-based capability within the Purview platform that detects, measures, and improves data accuracy, completeness, and consistency across enterprise systems through automated profiling, rule creation, and remediation tracking.
It enables organizations to assess data health at scale by scanning datasets for anomalies such as missing values, invalid formats, or outliers. Once detected, teams can define business rules-like requiring all customer emails to follow a standard format-and monitor compliance over time. This reduces manual review and prevents downstream errors in analytics, reporting, and AI models.
Data quality in Purview operates across structured and semi-structured sources including Azure SQL, Synapse, Power BI, and external databases. It integrates with existing data pipelines and supports continuous monitoring, making it suitable for regulated industries like finance and healthcare where data integrity is critical.
Purview Data Quality begins with discovery. When a dataset is added to the Purview catalog, the system automatically runs a profile analysis to extract metadata such as data types, value ranges, null rates, and uniqueness. This initial scan generates a baseline of data health.
From this baseline, users create quality rules. For example, a rule can flag any record where the "customer_age" field exceeds 120 years-an impossible value indicating data entry error. Rules can be based on thresholds (e.g., <5% missing values), regex patterns (e.g., valid email format), or custom logic using Python expressions.
Rules run continuously or on a schedule. When violations occur, Purview logs them and notifies stakeholders via alerts. A dashboard displays trend lines showing data quality scores over time, helping teams track improvement efforts. The system maintains audit trails for compliance purposes.
Microsoft Purview Data Quality uses four primary components:
There are three rule types:
| Type | Description | Example |
|---|---|---|
| Format Validation | Ensures data follows a specified structure (e.g., date format). | "Order_date" must be in YYYY-MM-DD format. |
| Business Logic | Applies domain-specific constraints. | "Total_sales" cannot be negative. |
| Statistical Threshold | Flags deviations from expected statistical behavior. | More than 10% of "product_price" entries above $10,000. |
Unlike basic data validation tools, Purview Data Quality doesn't just flag issues-it correlates them with data lineage. If a bad value originates from a specific ETL job, the tool traces it back to the source system, enabling faster root cause analysis.
Many assume Purview Data Quality is limited to detecting missing or malformed fields. That's incorrect. It identifies logical inconsistencies too-such as a customer's "last_contact_date" being later than their "account_creation_date," which violates business logic.
Another misconception is that setting up rules requires coding. While advanced users can write custom scripts, most rules are created through a visual interface using dropdowns and predefined templates. For example, selecting "Email Format" from a list applies a standard regex pattern without writing code.
A less obvious point: Purview doesn't fix data itself. It identifies problems and triggers workflows-either manually or through integration with Azure Logic Apps or Power Automate-to initiate corrections. This separation ensures control and traceability.
A national retail chain used Microsoft Purview Data Quality to address recurring issues in inventory reporting. Their warehouse management system exported daily stock counts to a central data lake, but discrepancies appeared in monthly sales forecasts.
After connecting the inventory dataset to Purview, they ran a profile analysis. The report showed 12% of "quantity_on_hand" values were negative-impossible under normal operations. They created a rule: "quantity_on_hand must be ≥ 0."
Over the next week, 47 records violated the rule. The system alerted the supply chain team. Investigation revealed a bug in the nightly sync process that reversed quantities during downtime. After fixing the ETL script, the violation rate dropped to 0.03%.
The team then set up a weekly report showing data quality scores. When the score dipped below 95%, the system sent an alert to the IT manager. This proactive monitoring reduced reconciliation time by 60% and improved forecast accuracy.
Best practices include starting with high-impact domains. Focus first on customer data, financial records, or product catalogs where poor quality leads to direct business risk. Avoid creating hundreds of rules upfront; begin with 3-5 critical ones and expand based on feedback.
Use consistent naming conventions for rules. Instead of "Rule_001," label them "Customer_Email_Format_Valid" so teams understand purpose at a glance. Enable versioning for rules to track changes over time.
According to Gartner (2023), organizations using automated data quality tools reduce data-related decision errors by up to 40%. Microsoft's own documentation confirms that customers achieve measurable improvements in data trustworthiness within 30 days of deploying Purview Data Quality.
For implementation, start with a pilot project on a single dataset. Measure baseline quality, deploy rules, and validate outcomes before scaling. Use the Purview portal's built-in templates for common scenarios like validating phone numbers or checking currency formatting.
Microsoft Purview Data Quality is a scalable, automated way to monitor and improve data reliability across hybrid environments. It combines profiling, rule-based validation, and lineage-aware tracking to surface issues early and support data stewardship.