Senior Data Engineer

Piper Companies is seeking a Senior Data Engineer to join a leading shipping and logistics organization. The Senior Data Engineer will handle operational aspects of the data ecosystem, including monitoring, automation, and data flow.

Responsibilities of the Senior Data Engineer include:

• Work with Azure Databricks to develop and deploy solutions for reporting and analytics.
• Create real-time streaming pipelines.
• Data movement and orchestration using Azure Data Factory.
• Work with internal customers and partners to gather requirements, build, and deploy robust models.
• Develop code to aggregate and transform data to ADLS curated zone.
• Provide ongoing support of Databricks code.

Qualifications for the Senior Data Engineer include:  

• 7+ years of experience in data strategies, analysis, architecture, and management
• Strong skills in Azure Data Factory, Databricks, SQL, Python (PySpark) and/or Scala
• Experience in building ETL/data warehouse transformation processes
• Hands-on experience with Azure Data Analytics platform (Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics)
• Experience with Apache Kafka and event-based data

Compensation for the Senior Data Engineer include:

• Salary: $140,000 - $160,000 -- Hourly: $73/hr - $77/hr
• Comprehensive Benefits: Medical, Dental, Vision, 401K, PTO, Sick Leave as required by law, and Holidays

This job opens for applications on April 21, 2025. Applications for this job will be accepted for at least 30 days from the posting date.

#LI-JT2 #LI-REMOTE

Keywords: Data Architecture, ETL, Data Warehousing, Big Data, SQL, Python, Spark, Hadoop, Data Pipelines, Cloud Computing, AWS, Azure, GCP, Data Modeling, Data Integration, Machine Learning, Data Governance, Data Security, NoSQL, Real-time Data Processing