Senior Data Engineer (Streaming and Big Data)

1 Month ago • 5 Years +

Job Summary

Job Description

As a Senior Data Engineer at Lytx, you will design, develop, deploy, and maintain Big Data pipelines in the cloud. You will be responsible for advising software developers and mentoring junior employees. Your responsibilities will include participating in the full software development lifecycle, contributing to the development of mission-critical data applications, researching and designing next-generation data engine platforms, developing long-term strategies for data pipelines and storage, and recommending database infrastructure and tools. You will also build and manage integrated and automated data pipelines using the Hadoop Ecosystem and streaming data pipelines for machine learning models. Furthermore, you will process documentation, create data flow diagrams, and mentor junior engineers and data analysts.
Must have:
  • 5 years of experience in Data Engineering or Software Engineering
  • 5 years experience developing data pipelines with Java, Python or Scala
  • Strong knowledge of Apache Hadoop ecosystem
  • Significant experience with Spark, Kafka, Nifi, and other streaming data tools
  • Experience deploying data pipelines in the cloud (AWS preferred)
  • Strong experience with Data Lake technologies and techniques
  • Solid understanding of T-SQL and ETL programming

Job Details

Lytx: Are you looking to take your career in Data Engineering to the next level? Do you enjoy building data pipelines that enable critical business insights? Are you an expert in designing and delivering scalable data streaming solutions? Lytx is looking for a Senior Data Engineer to join our growing team! You’ll know you’re right for this job if you enjoy designing, developing, deployment and maintaining Big Data pipelines in the Cloud. In addition, you intimately know the Data Engineering landscape and enjoy advising software developers and mentoring more junior employees.

What You'll Do:

  • Participate in the full software development lifecycle (requirements, design, code, unit test, deployment, sustaining)
  • Provide senior-level contributions to the development teams responsible for implementing mission critical data applications  
  • Participate in the research, design, and testing of next generation data engine platforms
  • Develop and guide long-term strategy for data pipelines and persistent and NoSQL data storage 
  • Evaluate and recommend database infrastructure and tools including cloud technologies  
  • Build integrated and automated data pipelines using the Hadoop Ecosystem (Spark, Kafka, Hive, Yarn, Oozie)  
  • Develop and manage steaming data pipelines used for training and developing machine learning models
  • Process documentation and data flow diagramming  
  • Mentor and coach junior engineers ,software developers, and data analysts      
  • Perform other related duties as assigned


What You'll Need:

  • Bachelor’s degree in Information Technology or relevant experience.
  • Minimum 5 years of experience in Data Engineering and/or Software Engineering
  • Minimum 5 years experience developing data pipelines with Java, Python or Scala (Real time pipelines preferred)
  • Strong knowledge of Apache Hadoop ecosystem such as HDFS, MapReduce, Sqoop, Yarn, Hive, and Oozie
  • Significant experience with Spark, Kafka, Nifi, and other streaming data tools
  • Experience deploying data pipelines in the cloud (AWS preferred)
  • Strong experience with Data Lake technologies and techniques
  • Solid understanding of T-SQL and ETL programming including SQL Server Integration Services (SSIS)  
  • Excellent conceptual, analytical, and problem-solving skills
  • Experience in a high growth, fast paced environment
  • Demonstrated record of delivering quality technology outcomes  
  • Strong organizational skills and a keen attention to detail
  • Self-motivated, engaged, and accountable

Innovation Lives Here


You go all in no matter what you do, and so do we. At Lytx, we’re powered by cutting-edge technology and Happy People. You want your work to make a positive impact in the world, and that’s what we do. Join our diverse team of hungry, humble and capable people united to make a difference.

Together, we help save lives on our roadways.

Find out how good it feels to be a part of an inclusive, collaborative team. We’re committed to delivering an environment where everyone feels valued, included and supported to do their best work and share their voices.

Lytx, Inc. is proud to be an equal opportunity/affirmative action employer and maintains a drug-free workplace. We’re committed to attracting, retaining and maximizing the performance of a diverse and inclusive workforce. EOE/M/F/Disabled/Vet.

Similar Jobs

GoTo Group - Lead Software Engineer - Identity Platform

GoTo Group

Bengaluru, Karnataka, India (Hybrid)
5 Months ago
Luxoft - Senior Mobile QA Automation

Luxoft

Pune, Maharashtra, India (On-Site)
6 Months ago
Exient - Senior Programmer

Exient

Floriana, Malta (On-Site)
10 Months ago
skillz - Software Engineer, Mobile React Native SDK

skillz

Bengaluru, Karnataka, India (On-Site)
3 Weeks ago
Next Level Business Services - Java Tech Lead

Next Level Business Services

Chicago, Illinois, United States (On-Site)
7 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Adyen - Kotlin Engineer

Adyen

Chicago, Illinois, United States (Hybrid)
2 Weeks ago
Epic Games - Engineering Lead

Epic Games

Vancouver, British Columbia, Canada (On-Site)
5 Months ago
bytedance - Software Engineer Intern (Recommendation Infrastructure)

bytedance

San Jose, California, United States (On-Site)
2 Months ago
Devoteam - Technical Lead (Java/Angular)

Devoteam

Morocco (On-Site)
1 Month ago
Netflix - Software Engineer - Finance and Tax Technology

Netflix

Warsaw, Masovian Voivodeship, Poland (Hybrid)
1 Month ago
PayPal - Manager, Software Engineering

PayPal

Bengaluru, Karnataka, India (Hybrid)
4 Days ago
Marsh McLennan - IT Application Support Analyst

Marsh McLennan

Mexico City, Mexico (Hybrid)
1 Month ago
endava - Senior Full Stack Developer (Java/Javascript)

endava

Bogotá, Bogota, Colombia (On-Site)
4 Days ago
Tencent - WeChat Tech Intern (Backend Developer)

Tencent

(On-Site)
7 Months ago
Adtran - Software Development Engineer II - Python Test

Adtran

Huntsville, Alabama, United States (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in India

Synechron - React JS Developer

Synechron

Pune, Maharashtra, India (On-Site)
1 Week ago
Illuminia - Digital Business Solution Architect

Illuminia

Bengaluru, Karnataka, India (On-Site)
3 Days ago
NVIDIA - Senior Boot Software Engineer

NVIDIA

Hyderabad, Telangana, India (On-Site)
1 Month ago
Thales - Customer Relation Expert

Thales

Noida, Uttar Pradesh, India (Hybrid)
3 Weeks ago
Capgemini - PCB Layout Engineer

Capgemini

Pune, Maharashtra, India (On-Site)
6 Days ago
Assystems - Draughts Person / Designer - Solar (Mechanical)

Assystems

Gurugram, Haryana, India (On-Site)
7 Months ago
The Walt Disney Company - Senior Systems Reliability Operations Engineer

The Walt Disney Company

Mumbai, Maharashtra, India (On-Site)
6 Months ago
OAO India - Digital Marketing Executive

OAO India

Pune, Maharashtra, India (Remote)
2 Years ago
Accenture - Application Developer

Accenture

Gurugram, Haryana, India (On-Site)
6 Days ago
Synechron - Murex Datamart Reporting Support Engineer

Synechron

Pune, Maharashtra, India (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Lytx® is a leading provider of video telematics, analytics, safety and productivity solutions for commercial and public sector fleets. Our unrivaled Driver Safety Program, powered by our best-in-class DriveCam® Event Recorder, is proven to help save lives and reduce risk. We harness the power of video to help clients see what happened in the past, manage their operations more efficiently in the present and improve driver behavior to change the future. Our customizable services and programs span driver safety, risk detection, fleet tracking, compliance and fuel management. Using the world’s largest driving database of its kind, along with proprietary machine vision and artificial intelligence technology, we help protect and connect thousands of fleets and more than one million drivers worldwide.

San Diego, California, United States (On-Site)

United States (Remote)

San Diego, California, United States (On-Site)

San Diego, California, United States (On-Site)

San Diego, California, United States (On-Site)

San Diego, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Lytx

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug