Data Engineer for Market Data Projects (with Streamlit Expertise)

7 Months ago • 3-5 Years • Data Analysis

Job Summary

Job Description

Develop a scalable data platform for market data from various sources (vendors, research providers, exchanges, PBs, web scraping). Make this data accessible to portfolio managers (systematic & fundamental), and enterprise functions (Operations, Risk, Trading, Compliance). Responsibilities include web scraping, building and maintaining the Snowflake and AWS-based platform, onboarding new data providers, data migration projects, and enhancing existing pipelines. Requires strong DevOps skills (Kubernetes, Docker, Jenkins), SQL, Python, and excellent communication.
Must have:
  • SQL
  • Python
  • Linux
  • Containerization (Docker, Kubernetes)
  • AWS
  • DevOps (K8s, Docker, Jenkins)
  • Good communication
Good to have:
  • Market Data Projects/Capital markets exp
  • Snowflake
  • Airflow
  • Streamlit

Job Details

Project description

Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics

Responsibilities

Web scraping using scripts/APIs/Tools

Help build and maintain greenfield data platform running on Snowflake and AWS

Understand the existing pipelines and enhance pipelines for the new requirements.

Onboarding new data providers

Data migration projects

Skills

Must have

SQL

Python

Linux

Containerization(Docker, Kubernetes)

Good communication skills

AWS

Strong on Dev ops side of things(K8s, Docker, Jenkins)

Nice to have

Market Data Projects/ Capital markets exp

Snowflake is a big plus

Airflow

Other

Languages

English: B2 Upper Intermediate

Seniority

Senior

Similar Jobs

Workato - Technical Consultant

Workato

Tokyo, Japan (On-Site)
1 Month ago
Springer Group - Integrated Solutions Manager-Marketing

Springer Group

Coimbatore, Tamil Nadu, India (On-Site)
4 Weeks ago
Modulate - Customer Support Manager

Modulate

Cambridge, England, United Kingdom (Hybrid)
2 Months ago
Coherent corp. - Senior Engineering Manager

Coherent corp.

Murrieta, California, United States (On-Site)
1 Month ago
HP - Saudi eCommerce Lead

HP

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
4 Weeks ago
zeta - Principal Data Scientist I

zeta

Bengaluru, Karnataka, India (On-Site)
3 Weeks ago
Granicus - Business Analyst

Granicus

Canada (Hybrid)
1 Month ago
Luxoft - IT Business Analyst

Luxoft

New Delhi, Delhi, India (Remote)
6 Months ago
PwC - Business Analyst Data Modelling-Associate  - Kolkata Y-14 - Technology Consulting

PwC

Kolkata, West Bengal, India (On-Site)
9 Months ago
kaizen gaming  - Senior Data Engineer, AI Applications

kaizen gaming

Lisbon, Lisbon, Portugal (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Ubisoft - Senior ML Data Scientist

Ubisoft

Montreal, Quebec, Canada (On-Site)
5 Months ago
Sprinkler - Senior Director- Product Engineering

Sprinkler

Gurugram, Haryana, India (On-Site)
11 Months ago
Google - Google Workspace Sales Specialist, New and Expansion

Google

Tokyo, Japan (On-Site)
2 Months ago
Qualcomm - Senior Staff Engineer - Windows

Qualcomm

Hyderabad, Telangana, India (On-Site)
4 Weeks ago
Alpha Sense - Staff Product Designer

Alpha Sense

New York, United States (Remote)
1 Month ago
Aspire - Senior Finance Manager

Aspire

United States (Hybrid)
3 Weeks ago
Stem,  Inc  - Marketing Coordinator

Stem, Inc

Gurugram, Haryana, India (On-Site)
2 Days ago
HCL Tech - Sr solution architect

HCL Tech

Noida, Uttar Pradesh, India (On-Site)
1 Month ago
Haleon - Analytical Scientist

Haleon

Suzhou, Jiangsu, China (On-Site)
4 Weeks ago
Collaborative Robotics - System Safety Engineer, Reliability

Collaborative Robotics

Santa Clara, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in Mexico City, Mexico City, Mexico

Marsh McLennan - Senior Software Engineer - .NET

Marsh McLennan

Mexico City, Mexico (Hybrid)
1 Month ago
Scopely - Data Analyst, Marketing Analytics

Scopely

Mexico City, Mexico City, Mexico (Hybrid)
4 Months ago
Univision - Sr. Editor

Univision

Mexico City, Mexico (On-Site)
1 Month ago
hogarth - CGI Lead / 3DS Max or Maya - V Ray

hogarth

Mexico City, Mexico (Hybrid)
3 Weeks ago
Thousand Eyes - Technical Account Manager

Thousand Eyes

Mexico City, Mexico City, Mexico (On-Site)
2 Weeks ago
Haleon - Sales Force Trainee

Haleon

Mexico City, Mexico (Hybrid)
1 Week ago
nissan - Quality Inspector

nissan

Jiutepec, Morelos, Mexico (On-Site)
2 Weeks ago
Marsh McLennan - Treasury Analyst

Marsh McLennan

Mexico City, Mexico (Hybrid)
2 Weeks ago
Nagarro - Senior Staff Engineer, PHP Magento

Nagarro

Mexico (Remote)
8 Months ago
Amber - Senior 2D Animator

Amber

Guadalajara, Jalisco, Mexico (Remote)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Luxoft - Regular Data Engineer

Luxoft

(Remote)
7 Months ago
Varonis  - Data Analytics Engineering

Varonis

Herzliya, Tel Aviv District, Israel (Hybrid)
5 Months ago
dun bradstreet - Principal, Data Scientist

dun bradstreet

Jacksonville, Florida, United States (On-Site)
1 Week ago
PayPal - Sr. Engineer -Data Infra - Venmo

PayPal

San Jose, California, United States (Hybrid)
3 Weeks ago
Nagarro - Senior Staff Engineer, Big Data

Nagarro

India (Remote)
8 Months ago
Apple - Hardware Development Engineer - Data Center Hardware

Apple

Austin, Texas, United States (On-Site)
1 Week ago
zoox - Data Engineer

zoox

Foster City, California, United States (Hybrid)
6 Years ago
Dream Sports - Software Development Engineer 3 - Machine Learning & Data Platform

Dream Sports

Mumbai, Maharashtra, India (On-Site)
4 Months ago
Jam City - Senior Data Analyst

Jam City

Toronto, Ontario, Canada (On-Site)
1 Year ago
blend - Senior Data Engineer

blend

Montevideo, Montevideo Department, Uruguay (Remote)
2 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Poland, Ohio, United States (Remote)

Kraków, Lesser Poland Voivodeship, Poland (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Ukrainka, Kyiv Oblast, Ukraine (Remote)

Bengaluru, Karnataka, India (On-Site)

Mississauga, Ontario, Canada (On-Site)

Toronto, Ontario, Canada (On-Site)

Poland, Ohio, United States (Remote)

Pune, Maharashtra, India (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug