Senior Data Engineer with Python and Databricks

1 Day ago • All levels

Job Summary

Job Description

As a Senior Data Engineer, you will collaborate with data scientists and analysts to understand data needs and build efficient data pipelines. Your responsibilities include designing, implementing, and maintaining data pipelines for data ingestion, processing, and transformation using cloud services and other technologies. You will create and maintain data solutions in Azure using services like Data Factory, Synapse, Databricks/Spark, Polars/Pandas, and Fabric. Ensuring data quality, integrity, and dependability by implementing data validation and cleansing procedures is crucial, as is enhancing the scalability, efficiency, and cost-effectiveness of data pipelines. Furthermore, you will be responsible for monitoring and resolving data pipeline issues, ensuring data consistency and availability.
Must have:
  • Create and maintain data pipelines.
  • Implement data validation and cleansing procedures.
  • Work with Azure services like Data Factory, Synapse, and Databricks/Spark.

Job Details

  • Work together with data scientists and analysts to understand the needs for data and create effective data pipelines
  • Solution design, implement, and maintain data pipelines for data ingestion, processing, and transformation using could services and other technologies
  • Create and maintain data solutions IN Azure, using any of the following: Data Factory, Synapse, Databricks/Spark, Polars/Pandas, Fabric 
  • Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
  • Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
  • Monitoring and resolving data pipeline problems, ensuring  consistency and availability of the data.

 

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in Timișoara, Timiș, Romania

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Iași, Iași County, Romania (On-Site)

Bucharest, Bucharest, Romania (On-Site)

São Paulo, State Of São Paulo, Brazil (On-Site)

Montevideo, Montevideo Department, Uruguay (On-Site)

Bogotá, Bogota, Colombia (On-Site)

Cali, Valle Del Cauca, Colombia (On-Site)

Bogotá, Bogota, Colombia (On-Site)

Bogota, Colombia (On-Site)

Bogotá, Bogota, Colombia (On-Site)

London, England, United Kingdom (On-Site)

View All Jobs

Get notified when new jobs are added by endava

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug