Senior/Lead Data Engineer

3 Months ago • 3-5 Years • Data Analysis

Job Summary

Job Description

N-iX is seeking a Senior/Lead Data Engineer to design complex ETL processes, build and maintain data pipelines using Python, improve data quality through anomaly detection, optimize data delivery, and automate manual processes. The role involves data modeling, collaborating with various teams, and partnering with stakeholders to resolve data-related technical issues. The company works with a major school transportation provider, handling data related to millions of student journeys daily. Responsibilities include improving efficiency and latency in data pipelines, building internal tools for data measurement, and designing infrastructure for scalability.
Must have:
  • Proficiency in Python and SQL
  • 3-5 years experience building Data Lakes
  • AWS Glue framework knowledge
  • Apache Kafka maintenance
  • Airflow for job scheduling
  • ETL process optimization
Good to have:
  • Schema and dimensional data design
  • Agile methodology experience
Perks:
  • Flexible working format
  • Competitive salary
  • Personalized career growth
  • Professional development tools
  • Education reimbursement
  • Corporate events

Job Details

N-iX is looking for a Senior/Lead Data Engineer to join our team!

About the project:

Our customer is the leading school transportation provider in North America, being the owner of more than a half of all yellow school buses in the United States. Every day, the company completes 5 million student journeys, moving more passengers than all U.S. airlines combined and delivers reliable, quality services for 1,100 school districts. 

N-iX has built a successful cooperation with the client delivering a range of complex initiatives. As a result, N-iX has been selected as a strategic long-term partner to drive the digital transformation on an enterprise level, fully remodeling the technology landscape for 55,000 employees and millions of people across North America. 

Responsibilities:

  • Design complex ETL processes of various data sources in the data warehouse
  • Build new and maintain existing data pipelines using Python to improve efficiency and latency
  • Improve data quality through anomaly detection by building and working with internal tools to measure data and automatically detect changes
  • Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
  • Perform data modeling and improve our existing data models for analytics
  • Collaborate with SMEs, architects, analysts, and others to build solutions that integrate data from many of our enterprise data sources
  • Partner with stakeholders, including data, design, product, and executive teams, and assist them with data-related technical issues

Requirements: 

  • Proficiency in Python and SQL
  • 3-5 years of commercial experience in building and maintaining a Data Lake
  • Good knowledge of AWS cloud services, including the Glue framework
  • Experience maintaining Apache Kafka
  • Knowledge of job scheduling and orchestration using Airflow
  • Well-versed in the optimization of ETL processes
  • Steady expertise in data processing tools, including Elasticsearch, Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, Presto.
  • Decent knowledge of CI/CD (Docker, Jenkins, Terraform, Git)
  • Good understanding of algorithms and data structures
  • Excellent communication skills, both written and verbal
  • Ukrainian language Advanced or higher.

Nice to have: 

  • Experience in the schema and dimensional data design
  • Collaboration within a scaled team, using Agile methodology

We offer*:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

*not applicable for freelancers

Similar Jobs

Tactic studios - Senior Engine Programmer

Tactic studios

Ontario, Canada (Remote)
3 Months ago
Cognite - Infrastructure Software Engineer

Cognite

Austin, Texas, United States (Hybrid)
1 Year ago
The Walt Disney Company - Account Manager

The Walt Disney Company

Antwerp, Flanders, Belgium (Hybrid)
3 Months ago
mighty koi - 3D Animator

mighty koi

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Months ago
Rackspace Technology - Full Stack Java Developer

Rackspace Technology

Giza, Giza Governorate, Egypt (On-Site)
1 Month ago
bytedance - Big Data Engineer - Data Lake / Feature Store

bytedance

Singapore (On-Site)
2 Months ago
Extreme Inc. - Data Analysis Engineer

Extreme Inc.

Tokyo, Japan (Hybrid)
3 Months ago
Zelis  - Data Science Engineer / Healthcare Data Analyst

Zelis

Hyderabad, Telangana, India (On-Site)
2 Months ago
Scopely - Senior Data Analyst, Marketing Analytics

Scopely

Mexico City, Mexico City, Mexico (Hybrid)
5 Months ago
Saviynt - Senior Data Analyst, Sales

Saviynt

United States (Remote)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Bytro - Game Developer - Javascript / HTML5 (f/m/x)

Bytro

Hamburg, Hamburg, Germany (Hybrid)
11 Months ago
Vigaet - Project Coordinator-Internship

Vigaet

Bengaluru, Karnataka, India (On-Site)
1 Year ago
MRI Software - Sales Development Representative

MRI Software

Sydney, New South Wales, Australia (On-Site)
2 Months ago
Varonis  - Manager of Customer Success

Varonis

United States (Remote)
3 Months ago
bytedance - Software Engineer, Architecture and Infrastructure

bytedance

San Jose, California, United States (On-Site)
9 Months ago
Scopely - Director of CRM

Scopely

California, United States (Hybrid)
5 Months ago
PwC - Manager - ESG & Social Impact

PwC

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (On-Site)
10 Months ago
Qualcomm - Emulation (FPGA Design) Engineer / Bluetooth - Staff

Qualcomm

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Blinkhealth - Executive Assistant

Blinkhealth

New York, United States (On-Site)
1 Month ago
USE Insider - Senior Content Writer - Remote

USE Insider

United States (Remote)
9 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Ukraine

plarium - C# Developer

plarium

Kyiv, Kyiv City, Ukraine (Remote)
2 Months ago
BetterMe - Automation QA Engineer (Android)

BetterMe

Ukraine (Remote)
2 Months ago
luxsoft - Full-stack Java Developer

luxsoft

Ukraine (Remote)
2 Months ago
nord current - Technical Designer

nord current

Dnipro, Dnipropetrovsk Oblast, Ukraine (On-Site)
3 Months ago
BetterMe - Tax Associate

BetterMe

Ukraine (Remote)
1 Month ago
Nium - Manual QA Engineer (Web)

Nium

Kyiv, Kyiv City, Ukraine (On-Site)
1 Month ago
BetterMe - Motion Designer/Video Editor

BetterMe

Ukraine (Remote)
2 Months ago
BetterMe - Graphic Designer

BetterMe

Ukraine (Remote)
2 Months ago
N-ix - Program Manager

N-ix

Ukraine (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Autodesk - Data Engineer

Autodesk

Toronto, Ontario, Canada (Hybrid)
2 Months ago
Next Level Business Services - Business Analyst - Mobility

Next Level Business Services

Collegeville, Pennsylvania, United States (On-Site)
9 Months ago
P99 soft - Data Architect (Snowflake)

P99 soft

Hyderabad, Telangana, India (On-Site)
3 Months ago
King - Senior Staff Software Engineer (Data)

King

San Francisco, California, United States (On-Site)
1 Month ago
luxsoft - Murex FO Business Analyst

luxsoft

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (On-Site)
2 Months ago
miniclip - Senior Data Analyst

miniclip

London, England, United Kingdom (On-Site)
3 Months ago
CAE - C-CST-115 Business Analyst Intern

CAE

Montreal, Quebec, Canada (Hybrid)
1 Month ago
Car Gurus - Principal Sales Commissions Analyst

Car Gurus

Boston, Massachusetts, United States (Hybrid)
2 Months ago
Kavalirio - Data Scientist

Kavalirio

Chantilly, Virginia, United States (On-Site)
3 Months ago
PwC - Senior Financial Data Analyst  | Deals (M&A) | Lyon | CDI | H/F

PwC

Lyon, Auvergne-Rhône-Alpes, France (On-Site)
10 Months ago

Get notifed when new similar jobs are uploaded