Senior Data Engineer

7 Months ago • 5 Years + • Data Analysis

Job Summary

Job Description

Hashlist seeks a Senior Data Engineer to design and build a scalable data platform for automotive industry projects. Responsibilities include building efficient data ingestion, storage, and streaming systems using big data technologies like Spark, Flink, Kafka, and AWS services (Kinesis, Lambda, EMR/EKS, MSK). The role involves collaborating with cross-functional teams, implementing data pipelines, ensuring data quality and security, and providing on-call support. Ideal candidates possess 5+ years of experience in cloud-native data streaming, AWS infrastructure management, and proficiency in languages such as Spark-Scala, PySpark, Python, or Java. Experience with geospatial data and containerization is a plus.
Must have:
  • 5+ years experience in cloud-native data streaming
  • Experience with Spark, Flink, Kafka, AWS services
  • AWS infrastructure optimization and security expertise
  • Proficiency in Spark-Scala, PySpark, Python, or Java
  • Strong SQL and NoSQL database knowledge
Good to have:
  • Geospatial data processing experience
  • Docker, Kubernetes, and Helm experience

Job Details

About the job

Hashlist is a platform for projects within the automotive industry.


We establish supplier relationships with automotive companies and provide a one-stop shop for candidates interested in working in the industry.


Are you ready to embark on this journey?


Our client is seeking a Senior Data Engineer. By applying, you gain access to the Hashlist network, unlocking opportunities in automotive projects across the industry.


Available locations: This position is open to candidates in Brazil and India (Bengaluru, Chennai & Pune).


Responsibilities:

  • Design and build a scalable data platform that can efficiently ingest, store, manage, and stream large volumes of data, simplifying analysis and enabling rapid development of high-quality data products and services.
  • Implement and test low-latency, real-time data pipelines using cutting-edge big data technologies to support data-as-a-service products.
  • Collaborate with cross-functional teams to clean, curate, and build end-to-end data pipelines based on requirements.
  • Design and implement privacy and security controls into the CI/CD of each data pipeline.
  • Ensure operational excellence by incorporating error handling, restartability, logging, monitoring, and alerting into the pipelines to maintain data consistency.
  • Drive continuous improvements in reliability, performance, scalability, and quality, owning the associated KPIs.
  • Provide on-call support for the platform and troubleshoot any issues that arise.


Qualifications:

  • 5+ years of experience in developing and operating cloud-native data streaming systems, with a focus on scalability, low latency, data privacy, and quality control.
  • Experience with cloud-native real-time streaming data technologies such as Spark, Flink, Kinesis, Lambda, Kafka, EMR/EKS platforms, and Lakehouse platforms (e.g., Delta.io).
  • 5+ years of experience in optimizing AWS data system architecture, including Infrastructure as Code, deployment automation, and security.
  • Proven experience in deploying and monitoring AWS infrastructure, particularly with MSK and EKS clusters in production environments.
  • Proficiency in programming languages such as Spark-Scala, Py-Spark, Python, or Java.
  • Strong knowledge of relational databases, SQL, SQL tuning, and NoSQL big data technologies.
  • Experience with geospatial data processing is a plus.
  • Experience in developing and deploying containerized applications using Docker, Kubernetes, and Helm.
  • Proven ability to build systems that monitor data quality and prevent data losses.
  • Strong understanding of data structures, algorithms, and operational excellence principles.
  • Curiosity and passion for learning, with a strong bias for action.


Next Steps:

  • Click "Apply."
  • We will review your application.
  • If qualified, you’ll be accepted into the Hashlist network and considered for this and other relevant projects.

Similar Jobs

Epic Games - Product Manager (BR & Reload)

Epic Games

Cary, North Carolina, United States (On-Site)
4 Months ago
Ceragon Networks - HRBP

Ceragon Networks

Romania (On-Site)
1 Month ago
Passive Logic - Senior Embedded Systems Engineer (Wireless Product Development)

Passive Logic

Holladay, Utah, United States (On-Site)
3 Months ago
Reddit - Staff Data Scientist, Ads

Reddit

United States (Remote)
2 Weeks ago
Toku - Senior HR Account Manager (HR Generalist)

Toku

United States (Remote)
5 Months ago
Synechron - Big Data Engineer

Synechron

Charlotte, North Carolina, United States (On-Site)
2 Weeks ago
The Walt Disney Company - Business Systems Analyst

The Walt Disney Company

Hong Kong (On-Site)
3 Months ago
moon shine game studios - Senior Software Engineer, Atlas Data Federation

moon shine game studios

United States (On-Site)
2 Weeks ago
Roblox - Senior/Principal Software Engineer, Data Engineering

Roblox

San Mateo, California, United States (On-Site)
1 Week ago
Trek - Marketing Data Analyst

Trek

Haryana, India (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Figma - Web Developer

Figma

San Francisco, California, United States (Remote)
2 Weeks ago
Ethos Life - Strategy & Business Operations, Senior Associate / Associate

Ethos Life

United States (Remote)
2 Months ago
LegalZoom - Senior Software Engineer

LegalZoom

Los Angeles, California, United States (Remote)
1 Week ago
Rippling - PEO Operations Analyst

Rippling

Bengaluru, Karnataka, India (On-Site)
1 Year ago
MRI Software - Asset Manager

MRI Software

Boston, Massachusetts, United States (Hybrid)
2 Months ago
Enphase Energy - Assistant Manager, Business Analytics

Enphase Energy

Bengaluru, Karnataka, India (On-Site)
6 Months ago
aspyr - Game Producer

aspyr

Austin, Texas, United States (On-Site)
2 Months ago
cirrus logic - Sr. Systems Engineer - Audio Amplifiers

cirrus logic

Austin, Texas, United States (Hybrid)
3 Weeks ago
CD PROJEKT RED - Expert Technical Artist - Graphics

CD PROJEKT RED

Warsaw, Masovian Voivodeship, Poland (Hybrid)
1 Month ago
legion - Product Manager

legion

Bucharest, Bucharest, Romania (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Pune, Maharashtra, India

warner bros games - Principal Engineer- Backend (MSC Team), Bangalore

warner bros games

Bengaluru, Karnataka, India (On-Site)
11 Months ago
Marvell - Technical Director, Physical Design

Marvell

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
[x]cube LABS - Associate Analyst - Design

[x]cube LABS

Hyderabad, Telangana, India (On-Site)
2 Weeks ago
Morning Star - Senior QA Automation Engineer

Morning Star

Delhi, India (Hybrid)
1 Month ago
Merkur gaming  - QA Automation (Python)

Merkur gaming

Noida, Uttar Pradesh, India (On-Site)
1 Month ago
Autodesk - Principal Engineer (Node.JS & React.JS)

Autodesk

Bengaluru, Karnataka, India (On-Site)
1 Year ago
GHX - Talent Acquisition Partner

GHX

Hyderabad, Telangana, India (On-Site)
4 Days ago
ShyftLabs - Technical Lead - Data Integrations

ShyftLabs

Noida, Uttar Pradesh, India (Hybrid)
1 Month ago
Alpha Sense - Senior Cloud Security Engineer

Alpha Sense

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Qube Cinema - Executive/ Sr. Executive - RMA Coordinator

Qube Cinema

Chennai, Tamil Nadu, India (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Nagarro - Senior Staff Consultant ,SAP Analytics Data Manageme

Nagarro

India (Remote)
8 Months ago
NVIDIA - Solutions Architect, Data Science

NVIDIA

Taipei City, Taiwan (On-Site)
5 Months ago
Brillio - Director, Data Science & AI

Brillio

Bengaluru, Karnataka, India (Hybrid)
2 Months ago
Actian - Sustenance Engineer - Actian Data Platform - Bangalore/Pune

Actian

Bengaluru, Karnataka, India (On-Site)
8 Months ago
Play Perfect - Senior Data Scientist

Play Perfect

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
1 Month ago
Axon - Sr. Business Systems Analyst - Salesforce

Axon

Scottsdale, Arizona, United States (On-Site)
1 Month ago
Tesla - Data Engineer

Tesla

North Holland, Netherlands (On-Site)
4 Months ago
Zscaler - Staff Data Center Operations Engineer

Zscaler

Sahibzada Ajit Singh Nagar, Punjab, India (Hybrid)
3 Weeks ago
Moloco - Data Scientist II, Growth Analytics

Moloco

London, England, United Kingdom (On-Site)
1 Month ago
PayPal - Manager, Data Science

PayPal

Bengaluru, Karnataka, India (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

Chennai, Tamil Nadu, India (Hybrid)

Pune, Maharashtra, India (Hybrid)

Bengaluru, Karnataka, India (Hybrid)

Pune, Maharashtra, India (Hybrid)

Pune, Maharashtra, India (Hybrid)

Pune, Maharashtra, India (Hybrid)

Bengaluru, Karnataka, India (Hybrid)

Pune, Maharashtra, India (Hybrid)

Pune, Maharashtra, India (Hybrid)

Bengaluru, Karnataka, India (Hybrid)

View All Jobs

Get notified when new jobs are added by Hashlist

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug