Data Engineer for Market Data Projects (with Streamlit Expertise)

5 Months ago • 3-5 Years • Data Analyst

Job Summary

Job Description

Develop a scalable data platform for market data from various sources (vendors, research providers, exchanges, PBs, web scraping). Make this data accessible to portfolio managers (systematic & fundamental), and enterprise functions (Operations, Risk, Trading, Compliance). Responsibilities include web scraping, building and maintaining the Snowflake and AWS-based platform, onboarding new data providers, data migration projects, and enhancing existing pipelines. Requires strong DevOps skills (Kubernetes, Docker, Jenkins), SQL, Python, and excellent communication.
Must have:
  • SQL
  • Python
  • Linux
  • Containerization (Docker, Kubernetes)
  • AWS
  • DevOps (K8s, Docker, Jenkins)
  • Good communication
Good to have:
  • Market Data Projects/Capital markets exp
  • Snowflake
  • Airflow
  • Streamlit

Job Details

Project description

Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics

Responsibilities

Web scraping using scripts/APIs/Tools

Help build and maintain greenfield data platform running on Snowflake and AWS

Understand the existing pipelines and enhance pipelines for the new requirements.

Onboarding new data providers

Data migration projects

Skills

Must have

SQL

Python

Linux

Containerization(Docker, Kubernetes)

Good communication skills

AWS

Strong on Dev ops side of things(K8s, Docker, Jenkins)

Nice to have

Market Data Projects/ Capital markets exp

Snowflake is a big plus

Airflow

Other

Languages

English: B2 Upper Intermediate

Seniority

Senior

Similar Jobs

Provenir - Senior Quality Assurance Automation Engineer

Provenir

Bengaluru, Karnataka, India (On-Site)
8 Months ago
Nagarro - Staff Engineer, .Net Fullstack

Nagarro

Bengaluru, Karnataka, India (On-Site)
6 Months ago
PwC - Manager-Data Engineer|Pune

PwC

Pune, Maharashtra, India (On-Site)
7 Months ago
The Walt Disney Company - Senior Machine Learning Engineer

The Walt Disney Company

New York, New York, United States (On-Site)
2 Weeks ago
Logifuture - Senior DevOps Engineer

Logifuture

Vojvodina, Serbia (Remote)
1 Month ago
Google - Associate, Analytical Lead, gReach Program for People with Disabilities

Google

Shanghai, Shanghai, China (On-Site)
2 Weeks ago
Second Dinner - Senior Data Engineer

Second Dinner

United States (Remote)
1 Month ago
Pluto7 - Data Scientist

Pluto7

Bengaluru, Karnataka, India (On-Site)
8 Months ago
Meta - Data Engineer, Product Analytics

Meta

Fremont, California, United States (On-Site)
5 Months ago
Playrix - Data QA Engineer

Playrix

Georgia (Remote)
6 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Saama Technologies,  Inc  - Senior Site Reliability Engineer

Saama Technologies, Inc

Chennai, Tamil Nadu, India (On-Site)
6 Months ago
Aisera Jobs - Senior Devops Engineer

Aisera Jobs

Hyderabad, Telangana, India (On-Site)
1 Day ago
Oriserve - Lead DevOps Engineer (5+ Yrs Exp)

Oriserve

Noida, Uttar Pradesh, India (On-Site)
5 Months ago
Nagarro - Principal Engineer, .Net Fullstack

Nagarro

India (Remote)
6 Months ago
Sporty Group - Software Engineering Team Lead - EU

Sporty Group

(Remote)
4 Weeks ago
Fi - Software Engineer - DevOps/Infra

Fi

Bengaluru, Karnataka, India (On-Site)
3 Years ago
Bigpoint - Associate DevOps Engineer (m/f/d) - #5720

Bigpoint

Hamburg, Hamburg, Germany (Remote)
8 Months ago
Minecast - Senior ML Engineer - NLP

Minecast

London, England, United Kingdom (Hybrid)
20 Hours ago
Aristocrat Gaming - QA Automation Engineer

Aristocrat Gaming

Barcelona, Catalonia, Spain (Hybrid)
1 Week ago
Offworld - DevOps Engineer

Offworld

New Westminster, British Columbia, Canada (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in Mexico City, Mexico City, Mexico

Google - Cloud Engineer II, AI/ML, Professional Services

Google

Mexico City, Mexico City, Mexico (On-Site)
2 Weeks ago
Warner Bros Games - Programmatic Coordinator

Warner Bros Games

Mexico City, Mexico City, Mexico (Hybrid)
2 Months ago
Scopely - Senior Data Analyst, Marketing Analytics

Scopely

Mexico City, Mexico City, Mexico (Hybrid)
2 Months ago
Hawk Eye Innovations - Baseball Tracking Systems Operator

Hawk Eye Innovations

Mexico City, Mexico City, Mexico (On-Site)
1 Week ago
techholding - Identity Infrastructure Engineer

techholding

Mexico City, Mexico (On-Site)
1 Day ago
HP - Product Life Cycle Project Manager

HP

Ciudad Guzmán, Jalisco, Mexico (On-Site)
7 Months ago
Lionbridge Games - Content Moderator

Lionbridge Games

Mexico City, Mexico City, Mexico (On-Site)
2 Weeks ago
Marsh McLennan - Industrial Engineer

Marsh McLennan

Mexico City, Mexico (Hybrid)
20 Hours ago
Nissan - ASESOR FT AIC

Nissan

Aguascalientes, Aguascalientes, Mexico (On-Site)
7 Months ago
Onward Search - Accessibility Consultant

Onward Search

Mexico City, Mexico City, Mexico (Remote)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

Playrix - Senior Data Analyst (Attribution)

Playrix

Ireland (Remote)
6 Months ago
ION - DCM Data Analyst - 1018

ION

Mumbai, Maharashtra, India (On-Site)
6 Months ago
Sovrun - Head of Engineering

Sovrun

Makati, Metro Manila, Philippines (Hybrid)
3 Weeks ago
Epic Games - Analytics Lead (Gameplay)

Epic Games

(On-Site)
3 Months ago
ByteDance - LLM Global Data - LLM Coding Trainer Intern - 2025 Start

ByteDance

Singapore (On-Site)
3 Months ago
Playrix - Senior Data Analyst (Game)

Playrix

Cyprus (Remote)
6 Months ago
Scopely - Data Analyst, Marketing Analytics

Scopely

Mexico City, Mexico City, Mexico (Hybrid)
2 Months ago
Playrix - Senior Data Analyst (Game)

Playrix

Montenegro (Remote)
6 Months ago
Google - Data Analyst, Trust and Safety, Search

Google

Seattle, Washington, United States (On-Site)
2 Days ago
Thatgamecompany - Product Data Scientist

Thatgamecompany

United States (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Poland, Ohio, United States (Remote)

Kraków, Lesser Poland Voivodeship, Poland (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Ukrainka, Kyiv Oblast, Ukraine (Remote)

Bengaluru, Karnataka, India (On-Site)

Mississauga, Ontario, Canada (On-Site)

Toronto, Ontario, Canada (On-Site)

Poland, Ohio, United States (Remote)

Pune, Maharashtra, India (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug