Data Engineer

2 Months ago • 5-7 Years • Data Analysis

Job Summary

Job Description

As a Data Engineer at IMC, you will be responsible for the secure reception and storage of trading-related data for the India teams, while also managing and improving shared data access and processing systems. This critical role supports trader analysis, simulation, reporting, and insights. You will manage distributed systems, storage infrastructure, and data processing platforms, focusing on systems automation and CI/CD for rapid deployment. Collaboration with other teams and staying updated with the latest industry technologies are also key responsibilities. This includes troubleshooting queries and implementing innovative solutions to continuously improve data infrastructure.
Must have:
  • Manage large-scale multi-petabyte data infrastructure
  • Advanced knowledge of Linux system administration
  • Expertise in Kafka, Spark, Cassandra/Scylla, or HDFS
  • Strong knowledge of Docker, Kubernetes, and Helm
  • Experience with data access technologies
  • Familiarity with workflow orchestration tools
  • Proficiency with CI/CD pipelines and Git

Job Details

Join our dynamic and high-impact Data team as a Data Engineer, where you'll be responsible for safely receiving and storing trading-related data for the India teams, as well as operating and improving our shared data access and data processing systems. This is a critical role in the organisation as the data platform drives a huge range of trader analysis, simulation, reporting and insights. The ideal candidate should have work experience in systems engineering, preferably with prior exposure to financial markets and with proven working knowledge in the fields of Linux administration, orchestration and automation tools, systems hardware architecture as well as storage and data protection technologies.

Your Core Responsibilities:

  • Manage and monitor all distributed systems, storage infrastructure, and data processing platforms, including HDFS, Kubernetes, Dremio, and in-house data pipelines
  • Drive heavy focus on systems automation and CI/CD to enable rapid deployment of hardware and software solutions
  • Collaborate closely with systems and network engineers, traders, and developers to support and troubleshoot their queries
  • Stay up to date with the latest technology trends in the industry; propose, evaluate, and implement innovative solutions

Your Skills and Experience:

  • 5–7 years of experience in managing large-scale multi-petabyte data infrastructure in a similar role
  • Advanced knowledge of Linux system administration and internals, with proven ability to troubleshoot issues in Linux environments
  • Deep expertise in at least one of the following technologies: Kafka, Spark, Cassandra/Scylla, or HDFS
  • Strong working knowledge of Docker, Kubernetes, and Helm
  • Experience with data access technologies such as Dremio and Presto
  • Familiarity with workflow orchestration tools like Airflow and Prefect
  • Exposure to cloud platforms such as AWS, GCP, or Azure
  • Proficiency with CI/CD pipelines and version control systems like Git
  • Understanding of best practices in data security and compliance
  • Demonstrated ability to solve problems proactively and creatively with a results-oriented mindset
  • Quick learner with excellent troubleshooting skills
  • High degree of flexibility and adaptability

About Us

IMC is a global trading firm powered by a cutting-edge research environment and a world-class technology backbone. Since 1989, we’ve been a stabilizing force in financial markets, providing essential liquidity upon which market participants depend. Across our offices in the US, Europe, Asia Pacific, and India, our talented quant researchers, engineers, traders, and business operations professionals are united by our uniquely collaborative, high-performance culture, and our commitment to giving back. From entering dynamic new markets to embracing disruptive technologies, and from developing an innovative research environment to diversifying our trading strategies, we dare to continuously innovate and collaborate to succeed.

 

Similar Jobs

Everi - Engineer Software Applications IV

Everi

Winnipeg, Manitoba, Canada (Hybrid)
1 Year ago
TAG - Social Media Content Manager

TAG

Toronto, Ontario, Canada (Hybrid)
1 Month ago
Ion - Senior Technical Consultant - Endur

Ion

Jersey City, New Jersey, United States (On-Site)
9 Months ago
Capco - Allegro Developer

Capco

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
Go Fund Me - Staff Data Engineer

Go Fund Me

Buenos Aires, Buenos Aires, Argentina (Hybrid)
2 Weeks ago
Comscore - Senior Data Analyst, Audience Analytics

Comscore

Pune, Maharashtra, India (On-Site)
1 Week ago
Demandbase - Software Engineer 2 (Data Platform)

Demandbase

Hyderabad, Telangana, India (On-Site)
2 Months ago
Unity - Senior Big Data & ML Engineer

Unity

Tel Aviv-Yafo, Tel Aviv District, Israel (Remote)
4 Months ago
Kavalirio - Data Scientist / Digital Commerce Marketing Analytics Consultant

Kavalirio

Atlanta, Georgia, United States (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

NVIDIA - Senior GPU Cluster Software Engineer

NVIDIA

Shanghai, Shanghai, China (On-Site)
6 Months ago
2K - Senior Product Manager - Horizon

2K

Austin, Texas, United States (On-Site)
2 Months ago
Rockstar Games - Production Coordinator

Rockstar Games

Leeds, England, United Kingdom (On-Site)
2 Months ago
Canonical - Senior Ubuntu Embedded IoT System Engineer

Canonical

(Remote)
2 Months ago
Codeway Studios - Data Analyst

Codeway Studios

Istanbul, İstanbul, Türkiye (On-Site)
2 Months ago
Tencent - Senior IT Operation Engineer

Tencent

Los Angeles, California, United States (On-Site)
5 Days ago
Progress - C++ Full Stack Engineer

Progress

Sofia, Sofia City Province, Bulgaria (Hybrid)
4 Months ago
eBay - MTS1, Services Engineer, SEO Graph

eBay

Bengaluru, Karnataka, India (Hybrid)
1 Month ago
Bluevine India - Data Architecture Lead

Bluevine India

Bengaluru, Karnataka, India (Hybrid)
2 Weeks ago
PwC - Senior Associate IT Assurance Services

PwC

Makati City, Metro Manila, Philippines (On-Site)
9 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Mumbai, Maharashtra, India

NCR Atleos - Senior Software Engineer (Java, J2EE)

NCR Atleos

Hyderabad, Telangana, India (Hybrid)
1 Month ago
bounteous - Wordpress Backend Developer

bounteous

India (Remote)
8 Months ago
Sony Music Career - Revenue Assurance Analyst

Sony Music Career

Mumbai, Maharashtra, India (Hybrid)
1 Month ago
Pluto7 - Data Scientist

Pluto7

Bengaluru, Karnataka, India (On-Site)
11 Months ago
Clearwater Analytics - Senior Associate, MS Operations Manager

Clearwater Analytics

Bengaluru, Karnataka, India (On-Site)
2 Weeks ago
NCR Voyix - Accounts Payable Specialist

NCR Voyix

Bengaluru, Karnataka, India (On-Site)
1 Week ago
Beta Craft - Full Stack Developer

Beta Craft

Pune, Maharashtra, India (On-Site)
4 Months ago
Qualcomm - GPU - Sr Staff Engineer

Qualcomm

Bengaluru, Karnataka, India (On-Site)
2 Months ago
PINKERTON | Comprehensive Risk Management - Senior GSOC Specialist

PINKERTON | Comprehensive Risk Management

Karnataka, India (On-Site)
8 Months ago
Assystems - Tunnel Design Engineer

Assystems

Mumbai, Maharashtra, India (On-Site)
8 Months ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

DataVisor - Solution Data Scientist (Fraud & AML)

DataVisor

Mountain View, California, United States (Hybrid)
3 Weeks ago
OKX - Senior Internal Audit Data Analyst

OKX

New York, United States (On-Site)
2 Weeks ago
Minecast - Principal, Corporate FP&A Analyst

Minecast

Lexington, Massachusetts, United States (Hybrid)
1 Week ago
Toast - Senior Data Engineer

Toast

Bengaluru, Karnataka, India (Hybrid)
2 Weeks ago
Varonis  - Data Analytics Engineer

Varonis

Herzliya, Tel Aviv District, Israel (Hybrid)
5 Months ago
Applike - Senior Data Engineer

Applike

Hamburg, Hamburg, Germany (Hybrid)
2 Months ago
PwC - Senior Associate, Data Science, Data and Analytics, Advisory

PwC

Bengaluru, Karnataka, India (On-Site)
1 Year ago
endava - Data Engineer

endava

Monterrey, Nuevo Leon, Mexico (On-Site)
1 Month ago
Cognite - Senior Data Engineer

Cognite

Pau, Nouvelle-Aquitaine, France (Hybrid)
2 Months ago
ShyftLabs - Data Engineer

ShyftLabs

Noida, Uttar Pradesh, India (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Zug, Zug, Switzerland (Hybrid)

Amsterdam, North Holland, Netherlands (On-Site)

Mumbai, Maharashtra, India (On-Site)

Zug, Zug, Switzerland (Hybrid)

Sydney, New South Wales, Australia (On-Site)

Mumbai, Maharashtra, India (On-Site)

Zug, Zug, Switzerland (On-Site)

New York, United States (On-Site)

Amsterdam, North Holland, Netherlands (On-Site)

Amsterdam, North Holland, Netherlands (On-Site)

View All Jobs

Get notified when new jobs are added by IMC

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug