Azure Data Engineer (Software Engineer/ Lead)

The role involves working in Software Engineering, focusing on the development, maintenance, and optimization of software solutions and applications. This includes applying scientific methods to analyze and solve software engineering problems, developing and applying software engineering practices, and using original thought and judgment. The engineer will build skills in their discipline, collaborate with other engineers and stakeholders, and leverage knowledge in Cloud Computing using Spark, Azure Databricks, and Azure Data Factory. Proficiency in Python/Scala, Spark/PySpark (Core and Streaming) for real-time or batch ingestion and transformation pipelines is required. The role also includes analyzing existing queries for performance improvements, developing procedures and scripts for data migration, providing scheduled management reporting, and investigating exceptions regarding asset movements. Working experience and strong knowledge in Databricks is a plus.