Senior/Lead Python Engineer (with Data Engineering experience)

2 Months ago • 3-5 Years • Data Analysis

Job Summary

Job Description

The Senior/Lead Python Engineer will be responsible for designing complex ETL processes, building and maintaining data pipelines using Python, and improving data quality through anomaly detection. They will also identify process improvements, perform data modeling, and collaborate with various teams to integrate data from enterprise sources. This role involves partnering with stakeholders, and assisting with data-related technical issues. The engineer is expected to have experience in building and maintaining a Data Lake, and knowledge of AWS cloud services including Glue framework, Apache Kafka, Airflow, Redis, Apache Spark, Apache Iceberg, and Athena. The candidate should also have excellent communication skills.
Must have:
  • Proficiency in Python and SQL
  • Experience building and maintaining a Data Lake
  • Knowledge of AWS cloud services (Glue)
  • Experience maintaining Apache Kafka
  • Knowledge of job scheduling and orchestration using Airflow
  • Expertise in data processing tools (Redis, Spark, Iceberg, Athena)
  • Well-versed in ETL process optimization
  • Understanding of algorithms and data structures
  • Excellent written and verbal communication skills
Good to have:
  • Experience in schema and dimensional data design
  • Collaboration within a scaled team, using Agile methodology
  • Knowledge of CI/CD (Docker, Cloud formation, Git)

Job Details

N-iX is looking for a Senior/Lead Python Engineer (with Data Engineering experience) to join our team!

About the project:

Our customer is the leading school transportation provider in North America, being the owner of more than a half of all yellow school buses in the United States. Every day, the company completes 5 million student journeys, moving more passengers than all U.S. airlines combined, and delivers reliable, quality services for 1,100 school districts. 

N-iX has built a successful cooperation with the client delivering a range of complex initiatives. As a result, N-iX has been selected as a strategic long-term partner to drive the digital transformation on an enterprise level, fully remodeling the technology landscape for 55,000 employees and millions of people across North America. 

Responsibilities:

  • Design complex ETL processes of various data sources in the data warehouse
  • Build new and maintain existing data pipelines using Python to improve efficiency and latency
  • Improve data quality through anomaly detection by building and working with internal tools to measure data and automatically detect changes
  • Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
  • Perform data modeling and improve our existing data models for analytics
  • Collaborate with SMEs, architects, analysts, and others to build solutions that integrate data from many of our enterprise data sources
  • Partner with stakeholders, including data, design, product, and executive teams, and assist them with data-related technical issues

Requirements: 

  • Proficiency in Python and SQL
  • 3-5 years of commercial experience in building and maintaining a Data Lake
  • Good knowledge of AWS cloud services, including the Glue framework with integration type of projects (2 years).
  • Experience maintaining Apache Kafka
  • Knowledge of job scheduling and orchestration using Airflow
  • Steady expertise in data processing tools, including Redis, Apache Spark, Apache Iceberg, and Athena.
  • Well-versed in the optimization of ETL processes
  • Good understanding of algorithms and data structures
  • Excellent communication skills, both written and verbal
  • Ukrainian language, Advanced or higher.

Nice to have: 

  • Experience in the schema and dimensional data design
  • Collaboration within a scaled team, using Agile methodology
  • Decent knowledge of CI/CD (Docker, Cloud formation, Git)

We offer*:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

*not applicable for freelancers

Similar Jobs

GoTo Group - Senior Data Warehouse Engineer - India

GoTo Group

Bengaluru, Karnataka, India (On-Site)
9 Months ago
Scopely - Finance Manager

Scopely

Culver City, California, United States (Hybrid)
2 Months ago
Rackspace Technology - Practice Manager, Data Science, AI and ML

Rackspace Technology

San Diego, California, United States (Remote)
4 Months ago
HoYoverse - Office Manager (Los Angeles, U.S.)

HoYoverse

Santa Monica, California, United States (On-Site)
1 Year ago
Ubisoft - Engine Programmer

Ubisoft

Mainz, Rhineland-Palatinate, Germany (Hybrid)
3 Months ago
London stock Exchange - Data Analyst with French / Portuguese

London stock Exchange

Gdynia, Pomeranian Voivodeship, Poland (On-Site)
1 Month ago
endava - Senior Data Engineer with Python and Databricks

endava

Pitești, Argeș, Romania (On-Site)
1 Month ago
Sporty - BI Analyst

Sporty

(On-Site)
1 Year ago
dYdX - Growth & Marketing Data Scientist Lead

dYdX

New York, United States (On-Site)
1 Month ago
Capgemini - Data Scientist

Capgemini

Pune, Maharashtra, India (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Forescout Technologies  Inc  - DevOps Engineer

Forescout Technologies Inc

Pune, Maharashtra, India (On-Site)
1 Month ago
Ion - Enterprise Account Manager

Ion

London, England, United Kingdom (On-Site)
5 Months ago
IMC - Senior Recruiter

IMC

Mumbai, Maharashtra, India (On-Site)
2 Months ago
Nintendo - Associate Software Engineer Tools (NST)

Nintendo

Redmond, Washington, United States (On-Site)
11 Months ago
TechBiz Global - Motion Graphics Designer (Relocation to UAE)

TechBiz Global

New Delhi, Delhi, India (On-Site)
1 Year ago
Scout - Senior Engineer, Chassis Component System Level Testing

Scout

Novi, Michigan, United States (On-Site)
1 Month ago
Nordson Corporation - Field Service Engineering Technician

Nordson Corporation

(Remote)
2 Months ago
Tesla - Sales Advisor

Tesla

Zaventem, Flanders, Belgium (On-Site)
5 Months ago
Bosch Group - SAP Fiori (Frontend) Developer Intern

Bosch Group

Ho Chi Minh City, Vietnam (On-Site)
1 Month ago
rivos - Thermal/Mechanical Lead

rivos

Santa Clara, California, United States (Hybrid)
6 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Ukraine

Luxoft - Data Engineer for Market Data Projects (with Streamlit Expertise)

Luxoft

Ukrainka, Kyiv Oblast, Ukraine (Remote)
8 Months ago
gismart - UA Manager (Paid Social)

gismart

Kyiv, Kyiv City, Ukraine (Hybrid)
2 Months ago
BetterMe - Junior Digital Marketing Specialist

BetterMe

Ukraine (On-Site)
2 Months ago
Playtika - QA Automation Engineer

Playtika

Ukraine (On-Site)
8 Months ago
N-ix - Middle/Senior 3D artist

N-ix

Ukraine (Remote)
1 Month ago
Playtika - Java Developer /Kyiv

Playtika

Ukraine (On-Site)
8 Months ago
Virtuos - PMO Producer

Virtuos

Ukraine (Hybrid)
3 Months ago
Pingle studios - Business Development Manager

Pingle studios

Kyiv, Kyiv City, Ukraine (Remote)
9 Months ago
Simcorp - Lead HR Business Partner

Simcorp

Kyiv, Kyiv City, Ukraine (Hybrid)
1 Month ago
playrix  - Senior Data Analyst (Game)

playrix

Ukraine (Remote)
9 Months ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Corsair - Business Intelligence Data Analyst

Corsair

Milpitas, California, United States (On-Site)
3 Months ago
Penn Interactive - Data Scientist

Penn Interactive

Philadelphia, Pennsylvania, United States (Remote)
2 Months ago
Expedia - Data Engineer III

Expedia

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Ubisoft - Senior ML Data Scientist

Ubisoft

Montreal, Quebec, Canada (On-Site)
6 Months ago
Bestex Research - Senior Data Engineer

Bestex Research

Bengaluru, Karnataka, India (On-Site)
5 Months ago
Addepar - Data Operations Analyst - Alternatives

Addepar

Edinburgh, Scotland, United Kingdom (On-Site)
2 Months ago
Spyke Games - Data Scientist

Spyke Games

İstanbul, Türkiye (On-Site)
9 Months ago
Blackshark - Senior Software Engineer - Data Plane Team

Blackshark

(Remote)
2 Months ago
Rackspace Technology - Sr Big Data Engineer - Oozie and Pig (GCP)

Rackspace Technology

United States (Remote)
3 Months ago
Well - Data Engineer

Well

(Remote)
2 Months ago

Get notifed when new similar jobs are uploaded