Senior Data Engineer (Streaming and Big Data)

2 Months ago • 5 Years + • Data Analysis

Job Summary

Job Description

As a Senior Data Engineer at Lytx, you will design, develop, deploy, and maintain Big Data pipelines in the cloud. You will be responsible for advising software developers and mentoring junior employees. Your responsibilities will include participating in the full software development lifecycle, contributing to the development of mission-critical data applications, researching and designing next-generation data engine platforms, developing long-term strategies for data pipelines and storage, and recommending database infrastructure and tools. You will also build and manage integrated and automated data pipelines using the Hadoop Ecosystem and streaming data pipelines for machine learning models. Furthermore, you will process documentation, create data flow diagrams, and mentor junior engineers and data analysts.
Must have:
  • 5 years of experience in Data Engineering or Software Engineering
  • 5 years experience developing data pipelines with Java, Python or Scala
  • Strong knowledge of Apache Hadoop ecosystem
  • Significant experience with Spark, Kafka, Nifi, and other streaming data tools
  • Experience deploying data pipelines in the cloud (AWS preferred)
  • Strong experience with Data Lake technologies and techniques
  • Solid understanding of T-SQL and ETL programming

Job Details

Lytx: Are you looking to take your career in Data Engineering to the next level? Do you enjoy building data pipelines that enable critical business insights? Are you an expert in designing and delivering scalable data streaming solutions? Lytx is looking for a Senior Data Engineer to join our growing team! You’ll know you’re right for this job if you enjoy designing, developing, deployment and maintaining Big Data pipelines in the Cloud. In addition, you intimately know the Data Engineering landscape and enjoy advising software developers and mentoring more junior employees.

What You'll Do:

  • Participate in the full software development lifecycle (requirements, design, code, unit test, deployment, sustaining)
  • Provide senior-level contributions to the development teams responsible for implementing mission critical data applications  
  • Participate in the research, design, and testing of next generation data engine platforms
  • Develop and guide long-term strategy for data pipelines and persistent and NoSQL data storage 
  • Evaluate and recommend database infrastructure and tools including cloud technologies  
  • Build integrated and automated data pipelines using the Hadoop Ecosystem (Spark, Kafka, Hive, Yarn, Oozie)  
  • Develop and manage steaming data pipelines used for training and developing machine learning models
  • Process documentation and data flow diagramming  
  • Mentor and coach junior engineers ,software developers, and data analysts      
  • Perform other related duties as assigned


What You'll Need:

  • Bachelor’s degree in Information Technology or relevant experience.
  • Minimum 5 years of experience in Data Engineering and/or Software Engineering
  • Minimum 5 years experience developing data pipelines with Java, Python or Scala (Real time pipelines preferred)
  • Strong knowledge of Apache Hadoop ecosystem such as HDFS, MapReduce, Sqoop, Yarn, Hive, and Oozie
  • Significant experience with Spark, Kafka, Nifi, and other streaming data tools
  • Experience deploying data pipelines in the cloud (AWS preferred)
  • Strong experience with Data Lake technologies and techniques
  • Solid understanding of T-SQL and ETL programming including SQL Server Integration Services (SSIS)  
  • Excellent conceptual, analytical, and problem-solving skills
  • Experience in a high growth, fast paced environment
  • Demonstrated record of delivering quality technology outcomes  
  • Strong organizational skills and a keen attention to detail
  • Self-motivated, engaged, and accountable

Innovation Lives Here


You go all in no matter what you do, and so do we. At Lytx, we’re powered by cutting-edge technology and Happy People. You want your work to make a positive impact in the world, and that’s what we do. Join our diverse team of hungry, humble and capable people united to make a difference.

Together, we help save lives on our roadways.

Find out how good it feels to be a part of an inclusive, collaborative team. We’re committed to delivering an environment where everyone feels valued, included and supported to do their best work and share their voices.

Lytx, Inc. is proud to be an equal opportunity/affirmative action employer and maintains a drug-free workplace. We’re committed to attracting, retaining and maximizing the performance of a diverse and inclusive workforce. EOE/M/F/Disabled/Vet.

Similar Jobs

Playtika - People Analyst

Playtika

Israel (On-Site)
8 Months ago
zoox - Software Engineer, Analytics Platform

zoox

Foster City, California, United States (Hybrid)
2 Weeks ago
GlobalStep - Chief Technology Officer

GlobalStep

Pune, Maharashtra, India (On-Site)
1 Month ago
PwC - Head of Digital Strategy – Senior Manager

PwC

Bangkok, Bangkok, Thailand (On-Site)
9 Months ago
DevRev - Customer Success Manager

DevRev

New York, United States (On-Site)
3 Weeks ago
The Walt Disney Company - Associate Business Systems Analyst

The Walt Disney Company

Hong Kong (On-Site)
5 Months ago
Rackspace Technology - Data Engineer III

Rackspace Technology

Vietnam (Remote)
3 Months ago
Ness - MS Fabric Data Engineer

Ness

Prague, Czechia (Remote)
3 Weeks ago
Amanotes - Product Data Analyst

Amanotes

Ho Chi Minh City, Ho Chi Minh City, Vietnam (On-Site)
3 Months ago
Razer - Data Analyst

Razer

Singapore (On-Site)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Coherent corp. - Equipment Engineering Technician Lead

Coherent corp.

Sherman, Texas, United States (On-Site)
3 Weeks ago
Rackspace Technology - Lead Engagement Manager

Rackspace Technology

Canada (Remote)
3 Weeks ago
Unity - Senior App Growth and Operations Manager

Unity

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
2 Months ago
Assist software  - Business Operations Specialist

Assist software

Suceava, Suceava County, Romania (Remote)
5 Months ago
NVIDIA - Solutions Architect, AI and ML

NVIDIA

Redmond, Washington, United States (On-Site)
3 Months ago
PhonePe - Site Reliability Engineer

PhonePe

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Techland - Senior System Designer - Live Operations

Techland

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)
1 Week ago
Safe security - Software Development Engineer II - Platform

Safe security

Bengaluru, Karnataka, India (On-Site)
4 Months ago
zeta - Senior Associate – People Tech & Analytics

zeta

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
Intel  - Technical Sales Specialist/Solutions Architect

Intel

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in India

Expedia - Senior Software Development Engineer

Expedia

Bengaluru, Karnataka, India (On-Site)
1 Month ago
London stock Exchange - Technology and Cyber GRC Principal Specialist

London stock Exchange

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Capgemini - QA Architect

Capgemini

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Paytm - Team Lead Sales

Paytm

Gwalior, Madhya Pradesh, India (On-Site)
1 Month ago
Oliver Agency - Senior Account Manager

Oliver Agency

Mumbai, Maharashtra, India (On-Site)
1 Week ago
Rackspace Technology - Professional Services Technical Cloud Project Manager

Rackspace Technology

India (Remote)
2 Weeks ago
Capgemini - P&C Auto Insurance

Capgemini

Noida, Uttar Pradesh, India (On-Site)
1 Month ago
Capgemini - Tibco

Capgemini

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Paytm - Messaging Solution - DM/Manager- Payments

Paytm

Noida, Uttar Pradesh, India (On-Site)
3 Months ago
Winzo - Data Engineer

Winzo

New Delhi, Delhi, India (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Autodesk - Senior Data Engineer

Autodesk

Pune, Maharashtra, India (On-Site)
3 Weeks ago
paxie games - Data Scientist

paxie games

Istanbul, İstanbul, Türkiye (On-Site)
2 Weeks ago
Apple - Cellular Data Science & Machine Learning Engineer

Apple

San Diego, California, United States (On-Site)
1 Month ago
Devoteam - Cloud Data Engineer SQL GCP

Devoteam

Beni-Mellal, Béni Mellal-Khenifra, Morocco (On-Site)
5 Months ago
Lionsgate - Project Manager - IT, PMO - Office of the CIO, Solutions & Data Analytics

Lionsgate

Santa Monica, California, United States (On-Site)
3 Weeks ago
Jam City - Data Engineer

Jam City

Montevideo, Montevideo Department, Uruguay (On-Site)
1 Month ago
Rackspace Technology - Practice Manager, Data Science, AI and ML

Rackspace Technology

San Diego, California, United States (Remote)
3 Months ago
Enphase Energy - Staff Data Analyst

Enphase Energy

Bengaluru, Karnataka, India (On-Site)
4 Months ago
GameJobs - Senior Data Scientist

GameJobs

São Paulo, State Of São Paulo, Brazil (On-Site)
1 Year ago
The Walt Disney Company - Business Systems Analyst

The Walt Disney Company

Hong Kong (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Lytx is the global leader in fleet management technologies. Our solutions harness the power of video to empower drivers and fleets to be safer and more efficient, productive, and profitable so they can thrive in today’s competitive environment. Through the Lytx platform, direct and reseller clients access our customizable services and programs spanning driver safety, risk detection, fleet tracking, compliance, preventative maintenance, and fuel management. Using the world’s largest driving database of its kind, along with proprietary machine vision and artificial intelligence technology, we help protect and connect thousands of fleets and 1.6 million drivers in more than 60 countries worldwide. Lytx is privately held and headquartered in San Diego, California. For more information, visit us at Lytx.com.


The SurfsightTM solution is Lytx's indirect market offering, available in North America and internationally. Strategic partners and resellers can use Surfsight's open API platform to easily add video to their telematics stack or utilize our stand-alone Surfsight Cloud dashboard to allow fleet managers to track vehicles, view risky and distracted driving events, retrieve videos from the field, and view and analyze data. The innovative technology in the Surfsight dash cam, powered by Lytx, uses robust machine vision and artificial intelligence to proactively detect and mitigate risk. It provides detailed analytics and real-time visibility into overall fleet performance, giving companies valuable data to help increase safety and savings through better fleet management. The solution offers an accessible entry point into video telematics without compromising on features, functionality, and configuration options. For more information visit https://www.lytx.com/en-us/surfsight.



Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (Hybrid)

Bengaluru, Karnataka, India (On-Site)

United States (Remote)

United States (Remote)

Bengaluru, Karnataka, India (On-Site)

San Diego, California, United States (On-Site)

United States (Remote)

San Diego, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Lytx, Inc.

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug