Data Quality Engineer

Piper Companies is seeking a Data Quality Engineer to design, implement, and scale enterprise data quality frameworks across modern data platforms. This role will focus on building automated validation pipelines, embedding quality into CI/CD processes, and ensuring data reliability across ingestion, transformation, and consumption layers. The ideal candidate brings strong SQL and programming expertise, deep knowledge of data quality dimensions, and hands-on experience with modern data stacks such as Fabric, Databricks, or Spark.

Responsibilities

• Design and implement automated data testing frameworks (e.g., PyTest, Great Expectations, dbt tests) to validate data pipelines at scale
• Develop advanced SQL queries for data profiling, reconciliation, validation, and anomaly detection
• Build and maintain data quality checks across ETL/ELT pipelines to ensure reliability across all stages of data movement
• Integrate data quality validations into CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, Airflow) to enforce automated quality gates
• Implement observability and monitoring solutions to detect anomalies, schema drift, and data freshness issues in production
• Perform root-cause analysis on data defects and partner with engineering teams to implement sustainable fixes
• Collaborate with data engineers, analysts, and stakeholders to define data quality standards and translate business rules into testable checks

Qualifications

• 6+ years of experience in data engineering or data quality engineering roles
• Strong SQL expertise with the ability to write complex queries for data validation and analysis
• Proficiency in Python (preferred) or Scala/Java for building data validation frameworks and automation
• Hands-on experience with data testing tools such as Great Expectations, dbt, or custom frameworks using PyTest
• Solid understanding of ETL/ELT pipelines and common data quality failure points
• Experience implementing data quality checks within CI/CD environments (Azure DevOps, GitHub Actions, Airflow, etc.)
• Familiarity with data quality dimensions: accuracy, completeness, consistency, timeliness, validity, and uniqueness
• Experience working with modern data platforms such as Microsoft Fabric, Databricks, or Spark
• Strong analytical and debugging skills with a focus on root-cause resolution
• Excellent communication skills and experience working across cross-functional teams

Compensation

• Salary: Competitive based on experience
• Benefits: Comprehensive benefits package (details not specified)

Keywords

Data Quality Engineering, SQL, Python, PyTest, Great Expectations, dbt, ETL, ELT, Data Pipelines, Data Validation, Data Observability, Monte Carlo, Soda, CI/CD, Azure DevOps, GitHub Actions, Airflow, Databricks, Microsoft Fabric, Apache Spark, Data Testing Frameworks, Data Governance, Root Cause Analysis, Bethesda MD

Tags: #LI-MJ1 #LI-ONSITE