Senior/Lead Python Engineer (with Data Engineering experience)

5 Minutes ago • 3-5 Years

Job Summary

Job Description

The Senior/Lead Python Engineer will be responsible for designing complex ETL processes, building and maintaining data pipelines using Python, and improving data quality through anomaly detection. They will also identify process improvements, perform data modeling, and collaborate with various teams to integrate data from enterprise sources. This role involves partnering with stakeholders, and assisting with data-related technical issues. The engineer is expected to have experience in building and maintaining a Data Lake, and knowledge of AWS cloud services including Glue framework, Apache Kafka, Airflow, Redis, Apache Spark, Apache Iceberg, and Athena. The candidate should also have excellent communication skills.
Must have:
  • Proficiency in Python and SQL
  • Experience building and maintaining a Data Lake
  • Knowledge of AWS cloud services (Glue)
  • Experience maintaining Apache Kafka
  • Knowledge of job scheduling and orchestration using Airflow
  • Expertise in data processing tools (Redis, Spark, Iceberg, Athena)
  • Well-versed in ETL process optimization
  • Understanding of algorithms and data structures
  • Excellent written and verbal communication skills
Good to have:
  • Experience in schema and dimensional data design
  • Collaboration within a scaled team, using Agile methodology
  • Knowledge of CI/CD (Docker, Cloud formation, Git)

Job Details

N-iX is looking for a Senior/Lead Python Engineer (with Data Engineering experience) to join our team!

About the project:

Our customer is the leading school transportation provider in North America, being the owner of more than a half of all yellow school buses in the United States. Every day, the company completes 5 million student journeys, moving more passengers than all U.S. airlines combined, and delivers reliable, quality services for 1,100 school districts. 

N-iX has built a successful cooperation with the client delivering a range of complex initiatives. As a result, N-iX has been selected as a strategic long-term partner to drive the digital transformation on an enterprise level, fully remodeling the technology landscape for 55,000 employees and millions of people across North America. 

Responsibilities:

  • Design complex ETL processes of various data sources in the data warehouse
  • Build new and maintain existing data pipelines using Python to improve efficiency and latency
  • Improve data quality through anomaly detection by building and working with internal tools to measure data and automatically detect changes
  • Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
  • Perform data modeling and improve our existing data models for analytics
  • Collaborate with SMEs, architects, analysts, and others to build solutions that integrate data from many of our enterprise data sources
  • Partner with stakeholders, including data, design, product, and executive teams, and assist them with data-related technical issues

Requirements: 

  • Proficiency in Python and SQL
  • 3-5 years of commercial experience in building and maintaining a Data Lake
  • Good knowledge of AWS cloud services, including the Glue framework with integration type of projects (2 years).
  • Experience maintaining Apache Kafka
  • Knowledge of job scheduling and orchestration using Airflow
  • Steady expertise in data processing tools, including Redis, Apache Spark, Apache Iceberg, and Athena.
  • Well-versed in the optimization of ETL processes
  • Good understanding of algorithms and data structures
  • Excellent communication skills, both written and verbal
  • Ukrainian language, Advanced or higher.

Nice to have: 

  • Experience in the schema and dimensional data design
  • Collaboration within a scaled team, using Agile methodology
  • Decent knowledge of CI/CD (Docker, Cloud formation, Git)

We offer*:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

*not applicable for freelancers

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in Ukraine

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!