Data Engineer for Market Data Projects (with Streamlit Expertise)

5 Months ago • 3-5 Years • Data Analyst

Job Summary

Job Description

Develop a scalable data platform for market data from various sources (vendors, research providers, exchanges, PBs, web scraping). Make this data accessible to portfolio managers (systematic & fundamental), and enterprise functions (Operations, Risk, Trading, Compliance). Responsibilities include web scraping, building and maintaining the Snowflake and AWS-based platform, onboarding new data providers, data migration projects, and enhancing existing pipelines. Requires strong DevOps skills (Kubernetes, Docker, Jenkins), SQL, Python, and excellent communication.
Must have:
  • SQL
  • Python
  • Linux
  • Containerization (Docker, Kubernetes)
  • AWS
  • DevOps (K8s, Docker, Jenkins)
  • Good communication
Good to have:
  • Market Data Projects/Capital markets exp
  • Snowflake
  • Airflow
  • Streamlit

Job Details

Project description

Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics

Responsibilities

Web scraping using scripts/APIs/Tools

Help build and maintain greenfield data platform running on Snowflake and AWS

Understand the existing pipelines and enhance pipelines for the new requirements.

Onboarding new data providers

Data migration projects

Skills

Must have

SQL

Python

Linux

Containerization(Docker, Kubernetes)

Good communication skills

AWS

Strong on Dev ops side of things(K8s, Docker, Jenkins)

Nice to have

Market Data Projects/ Capital markets exp

Snowflake is a big plus

Airflow

Other

Languages

English: B2 Upper Intermediate

Seniority

Senior

Similar Jobs

The Walt Disney Company - Software Engineer, Test

The Walt Disney Company

California, United States (On-Site)
1 Month ago
NVIDIA - Senior Site Reliability Engineer, Data Science and ML Platforms

NVIDIA

Shanghai, Shanghai, China (On-Site)
3 Months ago
PwC - Manager Architecte Cloud et Intégration | CDI | H/F

PwC

Neuilly-sur-Seine, Île-de-France, France (On-Site)
6 Months ago
ION - Principal Software Engineer, Italy

ION

Collecchio, Emilia-Romagna, Italy (On-Site)
6 Months ago
Nagarro - Associate Principal Engineer, .Net Fullstack

Nagarro

India (Remote)
6 Months ago
Dream Sports - Manager - Business Analytics

Dream Sports

Mumbai, Maharashtra, India (On-Site)
5 Months ago
Tencent - Data Analytics Intern

Tencent

Quebec, Canada (On-Site)
1 Month ago
Microsoft - Senior Software Engineering Manager

Microsoft

Noida, Uttar Pradesh, India (On-Site)
4 Days ago
Attentive - Senior Data Scientist/Applied Scientist, AI Products

Attentive

San Francisco, California, United States (Hybrid)
6 Months ago
Netflix - Senior Research Analyst, Consumer Insights

Netflix

(On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Acceldata - Senior SDET - ADOC

Acceldata

Bengaluru, Karnataka, India (On-Site)
7 Months ago
Aristocrat Gaming - DevOps Lead

Aristocrat Gaming

Montreal, Quebec, Canada (Hybrid)
2 Months ago
ARHS - Senior Cloud/DevOps Architect

ARHS

Luxembourg (On-Site)
6 Months ago
Accurate - Software Engineer - DOTNET

Accurate

Hyderabad, Telangana, India (Hybrid)
6 Months ago
ION - Senior Full-Stack Developer, New York

ION

New York, New York, United States (Hybrid)
6 Months ago
Info Stretch - Senior Engineer

Info Stretch

Pune, Maharashtra, India (On-Site)
5 Months ago
N-iX - Lead iOS Engineer

N-iX

Ukraine (Remote)
5 Days ago
NVIDIA - Technical Program Manager, Developer Infrastructure

NVIDIA

Redmond, Washington, United States (On-Site)
1 Month ago
Thatgamecompany - Technical Support Engineer - China

Thatgamecompany

Shanghai, Shanghai, China (On-Site)
4 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Mexico City, Mexico City, Mexico

Lionbridge Games - Inventory Project Manager

Lionbridge Games

Mexico City, Mexico City, Mexico (On-Site)
1 Month ago
Netflix - Title Marketing Manager, Mexico

Netflix

Mexico City, Mexico City, Mexico (On-Site)
3 Months ago
Google - Deal Pursuit Team Manager, Google Cloud

Google

Mexico City, Mexico City, Mexico (On-Site)
6 Days ago
Lionbridge Games - Community Moderator

Lionbridge Games

Mexico City, Mexico City, Mexico (On-Site)
1 Month ago
Google - Senior Business Data Scientist, Behavioral Economics

Google

Mexico City, Mexico City, Mexico (On-Site)
6 Days ago
Nissan - Analista Técnico Cuarto Central de Control

Nissan

Jiutepec, Morelos, Mexico (On-Site)
6 Months ago
Nagarro - Senior Staff Engineer, QA Automation

Nagarro

Mexico (Remote)
6 Months ago
Lionbridge Games - Localization Project Manager

Lionbridge Games

Mexico City, Mexico City, Mexico (On-Site)
2 Months ago
Google - Software Engineer III, Education and Activation, Core

Google

Mexico City, Mexico City, Mexico (On-Site)
1 Week ago
Google - Senior Technical Solutions Consultant, Carrier, Android

Google

Mexico City, Mexico City, Mexico (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

Universal Music - Full Stack Engineer

Universal Music

Alabama, United States (Remote)
1 Week ago
Tesla - Data Engineer Internship

Tesla

North Holland, Netherlands (On-Site)
2 Months ago
Deutsche Bank - BM Analyst

Deutsche Bank

Mumbai, Maharashtra, India (On-Site)
6 Months ago
Microsoft - Software Engineer (Data Engineer)

Microsoft

Hyderabad, Telangana, India (On-Site)
1 Week ago
Ciklum - Senior Data Scientist

Ciklum

Chennai, Tamil Nadu, India (Hybrid)
6 Months ago
N-iX - Senior Data Scientist

N-iX

Ukraine (Remote)
1 Week ago
Rackspace Technology - Data Science Trainee

Rackspace Technology

Vietnam (Hybrid)
4 Weeks ago
ByteDance - Senior Software Engineer, Database Acceleration

ByteDance

San Jose, California, United States (On-Site)
1 Week ago
Rackspace Technology - Data Architect (Azure and Databricks)

Rackspace Technology

Gurugram, Haryana, India (Remote)
3 Weeks ago
PwC - In_Manager_ Business Analyst_D&A_Advisory_Mumbai

PwC

Mumbai, Maharashtra, India (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Poland, Ohio, United States (Remote)

Kraków, Lesser Poland Voivodeship, Poland (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Ukrainka, Kyiv Oblast, Ukraine (Remote)

Bengaluru, Karnataka, India (On-Site)

Mississauga, Ontario, Canada (On-Site)

Toronto, Ontario, Canada (On-Site)

Poland, Ohio, United States (Remote)

Pune, Maharashtra, India (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug