Senior Data Engineer with Python and Databricks

As a Senior Data Engineer, you will collaborate with data scientists and analysts to understand data needs and build efficient data pipelines. Your responsibilities include designing, implementing, and maintaining data pipelines for data ingestion, processing, and transformation using cloud services and other technologies. You will create and maintain data solutions in Azure using services like Data Factory, Synapse, Databricks/Spark, Polars/Pandas, and Fabric. Ensuring data quality, integrity, and dependability by implementing data validation and cleansing procedures is crucial, as is enhancing the scalability, efficiency, and cost-effectiveness of data pipelines. Furthermore, you will be responsible for monitoring and resolving data pipeline issues, ensuring data consistency and availability.