Data Engineer for Market Data Projects (with Streamlit Expertise)

3 Months ago • 3-5 Years • Data Analyst

Job Summary

Job Description

Develop a scalable data platform for market data from various sources (vendors, research providers, exchanges, PBs, web scraping). Make data accessible to PMs, Ops, Risk, Trading, and Compliance teams. Responsibilities include web scraping, building and maintaining a Snowflake and AWS-based platform, onboarding new data providers, and managing data migration projects. Enhance existing pipelines and develop internal data products and analytics. Strong DevOps skills (K8s, Docker, Jenkins) are essential.
Must have:
  • SQL
  • Python
  • Linux
  • Containerization (Docker, Kubernetes)
  • AWS
  • Good communication
  • Strong DevOps
Good to have:
  • Market Data Projects
  • Capital markets exp
  • Snowflake
  • Airflow
  • Streamlit

Job Details

Project description

Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics

Responsibilities

Web scraping using scripts/APIs/Tools

Help build and maintain greenfield data platform running on Snowflake and AWS

Understand the existing pipelines and enhance pipelines for the new requirements.

Onboarding new data providers

Data migration projects

Skills

Must have

SQL

Python

Linux

Containerization(Docker, Kubernetes)

Good communication skills

AWS

Strong on Dev ops side of things(K8s, Docker, Jenkins)

Nice to have

Market Data Projects/ Capital markets exp

Snowflake is a big plus

Airflow

Other

Languages

English: B2 Upper Intermediate

Seniority

Senior

Similar Jobs

HP - Front End Software Engineer

HP

Porto Alegre, State Of Rio Grande Do Sul, Brazil (On-Site)
• 4 Months ago
Enphase Energy - Sr. Software Engineer (QA Lead)

Enphase Energy

Bengaluru, Karnataka, India (On-Site)
• 1 Month ago
SciPlay - PHP Backend Developer

SciPlay

Kyiv, Kyiv City, Ukraine (Remote)
• 1 Month ago
Fortis Games - Staff QA Engineer

Fortis Games

Brazil (On-Site)
• 3 Months ago
EXUSIA - Ab Initio Technical Leads ( US - H1B Visa Holders)

EXUSIA

India (Remote)
• 4 Months ago
Intel Corporation - Data Architect

Intel Corporation

Phoenix, Arizona, United States (Hybrid)
• 2 Months ago
Tesla - Data Engineering Internship - New Product Introduction, BOM Management

Tesla

Brandenburg, Germany (On-Site)
• 1 Week ago
Extreme Network - Director, Data Governance, Enterprise Data and Analytics

Extreme Network

North Carolina, United States (Remote)
• 4 Months ago
Kwalee - Product Analyst

Kwalee

Bengaluru, Karnataka, India (On-Site)
• 3 Weeks ago
VGW - Business Intelligence Analyst

VGW

Perth, Western Australia, Australia (On-Site)
• 4 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Reversing Labs - DevOps Engineer- CI/CD & Cloud Specialist

Reversing Labs

Zagreb, Croatia (Hybrid)
• 4 Months ago
Nagarro - Senior QA Lead (all genders)

Nagarro

Austria (Remote)
• 4 Months ago
Sinch - DevOps Engineer (Email)

Sinch

Uttar Pradesh, India (Hybrid)
• 2 Months ago
CloudLinux - Principal Software Engineer / Product Owner (worldwide remote, work anywhere)

CloudLinux

Valencian Community, Spain (Remote)
• 2 Months ago
Bohemia Interactive - Senior QA Automation Engineer

Bohemia Interactive

Prague, Prague, Czechia (On-Site)
• 2 Weeks ago
Nielsen Holdings - Software Engineer (Java/Scala, SQL, AWS, Spark on Kubernetes)

Nielsen Holdings

Bengaluru, Karnataka, India (Hybrid)
• 4 Months ago
Teradata - Senior Cloud Engineer

Teradata

Pune, Maharashtra, India (On-Site)
• 3 Months ago
TOPPAN Edge  Inc  - Senior QA Engineer I

TOPPAN Edge Inc

Bengaluru, Karnataka, India (On-Site)
• 3 Months ago
NVIDIA - Senior Architect - NVIDIA Architecture

NVIDIA

Santa Clara, California, United States (On-Site)
• 1 Month ago
Luxoft - Senior Java Developer

Luxoft

Tampa, Florida, United States (On-Site)
• 2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Brazil, Indiana, United States

Meta - AI Research Scientist, Language - Generative AI

Meta

Seattle, Washington, United States (On-Site)
• 3 Months ago
The Walt Disney Company - Principal Machine Learning Engineer

The Walt Disney Company

San Francisco, California, United States (On-Site)
• 2 Months ago
Next Level Business Services - DevOps Consultant

Next Level Business Services

San Diego, California, United States (On-Site)
• 4 Months ago
Gamefam - Executive Producer

Gamefam

United States (Remote)
• 1 Week ago
Life church - Life.Church Campus Internship

Life church

United States (On-Site)
• 4 Months ago
CD PROJEKT RED - IT Director

CD PROJEKT RED

Boston, Massachusetts, United States (On-Site)
• 3 Months ago
Meta - Research Scientist, Machine Learning (PhD)

Meta

Sunnyvale, California, United States (On-Site)
• 3 Months ago
Overwolf - Publisher Success Manager - Nitro

Overwolf

Chicago, Illinois, United States (On-Site)
• 1 Month ago
Marvell - Facilities Commodity Manager

Marvell

Santa Clara, California, United States (Hybrid)
• 3 Months ago
Onward Search - Copy Editor

Onward Search

Los Angeles, California, United States (Hybrid)
• 5 Days ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

Hawk Eye Innovations - Sports Data Quality Specialist

Hawk Eye Innovations

Sofia, Sofia City Province, Bulgaria (On-Site)
• 3 Weeks ago
Next Level Business Services - Business Intelligence

Next Level Business Services

Philadelphia, Pennsylvania, United States (On-Site)
• 4 Months ago
prizepicks - Engineering Manager — Data Science Engineering

prizepicks

Atlanta, Georgia, United States (Remote)
• 1 Month ago
UXBERT Labs - Senior Business Analyst

UXBERT Labs

Riyadh, Riyadh Province, Saudi Arabia (Hybrid)
• 1 Month ago
Tencent - Data Scientist

Tencent

Palo Alto, California, United States (On-Site)
• 6 Months ago
ByteDance - System Architect Software Engineer - Data Platform

ByteDance

Singapore (On-Site)
• 3 Months ago
Sony India Software Centre - BW, BPC skilled resource -8~10 years of experience-4

Sony India Software Centre

Bengaluru, Karnataka, India (On-Site)
• 4 Months ago
PwC - Data Protection Expert

PwC

Prague, Prague, Czechia (Hybrid)
• 2 Months ago
ION - Junior Consultant - Risk Advisory, Italy

ION

Milan, Lombardy, Italy (On-Site)
• 4 Months ago
Head Digital Works - Data Scientist

Head Digital Works

Hyderabad, Telangana, India (On-Site)
• 7 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Gothenburg, Västra Götaland County, Sweden (On-Site)

New Delhi, Delhi, India (Remote)

Poland, Ohio, United States (Remote)

Kraków, Lesser Poland Voivodeship, Poland (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Ukrainka, Kyiv Oblast, Ukraine (Remote)

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (On-Site)

Bengaluru, Karnataka, India (On-Site)

Mississauga, Ontario, Canada (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug