Drive data innovation as a Senior Data Engineer. This position specializes in building efficient data infrastructures using Databricks, Apache Spark, and Python on Microsoft Azure platforms.
You will take ownership of the entire Lakehouse architecture, focusing on performance tuning and data quality implementation. The role includes coordinating with stakeholders for data governance and security while mentoring emerging talent. Utilize your expertise to design and implement robust analytics-ready data models effectively.
Key Responsibilities:
• Design and maintain data pipelines on Databricks
• Create Lakehouse architectures with Delta technologies
• Execute data ingestion workflows using Azure Data Factory
• Implement observability and quality checks in data processes
• Optimize Spark workload performance for cost efficiency
Requirements:
• 5+ years in data engineering experience required
• Direct experience with Databricks and Apache Spark for 2 years