Data Engineer

This role involves working in Software Engineering, focusing on the development, maintenance, and optimization of software solutions. The Data Engineer will apply scientific methods to solve software engineering problems, develop and apply software engineering practices, and collaborate with cross-functional teams. Key responsibilities include designing and developing data ingestion pipelines and working on ETL/ELT processes for large-scale data systems.

Must Have

• Design and develop data ingestion pipelines
• Work on ETL/ELT processes for large-scale data systems
• Collaborate with cross-functional teams to ensure data quality and performance
• Python
• AWS
• Databricks
• PySpark
• SQL

Job Description

Works in the area of Software Engineering, which encompasses the development, maintenance and optimization of software solutions/applications.

1. Applies scientific methods to analyse and solve software engineering problems.

2. He/she is responsible for the development and application of software engineering practice and knowledge, in research, design, development and maintenance.

3. His/her work requires the exercise of original thought and judgement and the ability to supervise the technical and administrative work of other software engineers.

4. The software engineer builds skills and expertise of his/her software engineering discipline to reach standard software engineer skills expectations for the applicable role, as defined in Professional Communities.

5. The software engineer collaborates and acts as team player with other software engineers and stakeholders.

Key Responsibilities:

• Design and develop data ingestion pipelines.
• Work on ETL/ELT processes for large-scale data systems.
• Collaborate with cross-functional teams to ensure data quality and performance.

---

Required Skills:

• Python
• AWS
• Databricks
• PySpark
• SQL