Data Engineer for Market Data Projects (with Streamlit Expertise)

8 Months ago • 3-5 Years • Data Analysis

Job Summary

Job Description

Develop a scalable data platform for market data from various sources (vendors, research providers, exchanges, PBs, web scraping). Make data accessible to PMs, Ops, Risk, Trading, and Compliance teams. Responsibilities include web scraping, building and maintaining a Snowflake and AWS-based platform, onboarding new data providers, and managing data migration projects. Enhance existing pipelines and develop internal data products and analytics. Strong DevOps skills (K8s, Docker, Jenkins) are essential.
Must have:
  • SQL
  • Python
  • Linux
  • Containerization (Docker, Kubernetes)
  • AWS
  • Good communication
  • Strong DevOps
Good to have:
  • Market Data Projects
  • Capital markets exp
  • Snowflake
  • Airflow
  • Streamlit

Job Details

Project description

Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics

Responsibilities

Web scraping using scripts/APIs/Tools

Help build and maintain greenfield data platform running on Snowflake and AWS

Understand the existing pipelines and enhance pipelines for the new requirements.

Onboarding new data providers

Data migration projects

Skills

Must have

SQL

Python

Linux

Containerization(Docker, Kubernetes)

Good communication skills

AWS

Strong on Dev ops side of things(K8s, Docker, Jenkins)

Nice to have

Market Data Projects/ Capital markets exp

Snowflake is a big plus

Airflow

Other

Languages

English: B2 Upper Intermediate

Seniority

Senior

Similar Jobs

bytedance - Research Scientist (Multimedia Streaming)

bytedance

San Diego, California, United States (On-Site)
2 Months ago
bytedance - Software Engineer, Multi Cloud CDN

bytedance

San Jose, California, United States (On-Site)
3 Months ago
Univision - Editor

Univision

Dallas, Texas, United States (On-Site)
1 Year ago
Marsh McLennan - Client Service Manager

Marsh McLennan

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (Hybrid)
1 Year ago
Interactive Brokers - Clearing Operations Analyst

Interactive Brokers

Tokyo, Japan (On-Site)
1 Month ago
Tide - Staff Data Scientist

Tide

Hyderabad, Telangana, India (On-Site)
3 Weeks ago
Roof Stacks - Mobile Banking Senior Business Analyst

Roof Stacks

Istanbul, İstanbul, Türkiye (Hybrid)
7 Months ago
Luxoft - Data Engineer (financial/investment domain)

Luxoft

Singapore, Singapore (On-Site)
8 Months ago
Social Discovery Ventures - Data Engineer

Social Discovery Ventures

Tbilisi, Tbilisi, Georgia (Remote)
3 Weeks ago
Ion - Senior Data Engineer, Risk Technology

Ion

Mexico City, Mexico (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Nagarro - SAP Cloud Senior Project Manager

Nagarro

Germany (Remote)
6 Months ago
endava - Senior Business Analyst

endava

Brașov, Brașov, Romania (On-Site)
2 Months ago
Merkur gaming  - Angular Developer

Merkur gaming

Noida, Uttar Pradesh, India (On-Site)
3 Weeks ago
truecaller - Senior Staff Data Engineer II

truecaller

Stockholm, Stockholm County, Sweden (On-Site)
2 Months ago
Jam City - Creative Marketing Producer

Jam City

Culver City, California, United States (Remote)
2 Weeks ago
Sawhorse Productions - Director of Finance and Accounting

Sawhorse Productions

Los Angeles, California, United States (Remote)
2 Months ago
Media molecule - Senior Designer (Environments)

Media molecule

London, England, United Kingdom (Hybrid)
4 Months ago
Mcdonalds - Senior Technical Analyst

Mcdonalds

London, England, United Kingdom (On-Site)
6 Days ago
Tennr - Backend Software Engineer

Tennr

New York, New York, United States (On-Site)
2 Weeks ago
Balbix - Staff AI Engineer

Balbix

Bengaluru, Karnataka, India (On-Site)
9 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Brazil, Indiana, United States

Apple - System Integration Lead

Apple

Austin, Texas, United States (On-Site)
1 Month ago
King - Staff Platform Solutions Engineer

King

New York, United States (On-Site)
1 Month ago
Decagon - Engineering Manager, Agent Software Engineering

Decagon

New York, New York, United States (On-Site)
1 Month ago
Roblox - Senior Engineering Director, Developer Monetization

Roblox

San Mateo, California, United States (On-Site)
1 Month ago
Palo Alto Networks - Principal DevOps Engineer

Palo Alto Networks

Santa Clara, California, United States (On-Site)
4 Weeks ago
Apple - Senior Data Engineer, Tools & Infrastructure

Apple

San Diego, California, United States (On-Site)
2 Months ago
Epic Games - User Experience Data Research Director

Epic Games

Cary, North Carolina, United States (On-Site)
4 Months ago
Next Level Business Services - Hadoop AWS Developer

Next Level Business Services

Beaverton, Oregon, United States (On-Site)
9 Months ago
Harvey - Applied AI Engineer

Harvey

San Francisco, California, United States (On-Site)
1 Week ago
Granicus - Strategic Account Manager

Granicus

Seattle, Washington, United States (Remote)
4 Months ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

luxsoft - Data Architect

luxsoft

Milton Keynes, England, United Kingdom (On-Site)
1 Month ago
Comscore - Data Analyst

Comscore

Reston, Virginia, United States (Remote)
1 Year ago
Nagarro - Associate Staff Consultant, Business Analyst

Nagarro

India (Remote)
9 Months ago
Syniverse - Data Engineer I

Syniverse

Bengaluru, Karnataka, India (On-Site)
1 Year ago
Cineplex - Analyst, Data & Insights

Cineplex

Toronto, Ontario, Canada (Hybrid)
1 Week ago
Jane Street - Data Centre Engineer

Jane Street

Mumbai, Maharashtra, India (On-Site)
2 Months ago
King - Senior Staff Software Engineer (Data)

King

San Francisco, California, United States (On-Site)
1 Month ago
Spaulding Ridge - Data Solutions Architect

Spaulding Ridge

Chicago, Illinois, United States (On-Site)
3 Weeks ago
Eqvilent - C++ Software Engineer (Market Data)

Eqvilent

(Remote)
2 Months ago
Epic Games - Principal Data Analyst, Ecosystem Economy & UGC

Epic Games

Cary, North Carolina, United States (On-Site)
7 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Poland, Ohio, United States (Remote)

New Delhi, Delhi, India (Remote)

Toronto, Ontario, Canada (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Gurugram, Haryana, India (On-Site)

Gurugram, Haryana, India (On-Site)

Gurugram, Haryana, India (On-Site)

Gurugram, Haryana, India (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug