Data Engineer for Market Data Projects (with Streamlit Expertise)

3 Months ago • 3-5 Years • Data Analyst

Job Summary

Job Description

Develop a scalable data platform for market data from various sources (vendors, research providers, exchanges, PBs, web scraping). Make this data accessible to portfolio managers, operations, risk, trading, and compliance teams. Responsibilities include web scraping, building and maintaining a Snowflake/AWS-based platform, enhancing existing pipelines, onboarding new data providers, and managing data migration projects. Strong DevOps skills (Kubernetes, Docker, Jenkins) are crucial. Experience with Streamlit is a plus.
Must have:
  • SQL
  • Python
  • Linux
  • Containerization (Docker, Kubernetes)
  • AWS
  • DevOps (K8s, Docker, Jenkins)
  • Good communication
Good to have:
  • Market Data Projects/Capital markets exp
  • Snowflake
  • Airflow

Job Details

Project description

Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics

Responsibilities

Web scraping using scripts/APIs/Tools

Help build and maintain greenfield data platform running on Snowflake and AWS

Understand the existing pipelines and enhance pipelines for the new requirements.

Onboarding new data providers

Data migration projects

Skills

Must have

SQL

Python

Linux

Containerization(Docker, Kubernetes)

Good communication skills

AWS

Strong on Dev ops side of things(K8s, Docker, Jenkins)

Nice to have

Market Data Projects/ Capital markets exp

Snowflake is a big plus

Airflow

Other

Languages

English: B2 Upper Intermediate

Seniority

Senior

Similar Jobs

Assystems - Lead Developer -  H/F

Assystems

Lyon, Auvergne-Rhône-Alpes, France (Hybrid)
• 3 Months ago
Luxoft - Senior Mobile QA Automation

Luxoft

Pune, Maharashtra, India (On-Site)
• 3 Months ago
Electronic Arts - Senior Software Engineer - .NET, Game Creation

Electronic Arts

Orlando, Florida, United States (Remote)
• 5 Months ago
Nagarro - Staff Engineer, Java Fullstack

Nagarro

Canada (Remote)
• 4 Months ago
Hitachi - Azure Developer

Hitachi

Hyderabad, Telangana, India (Remote)
• 4 Months ago
Varonis  - BI Developer

Varonis

Herzliya, Tel Aviv District, Israel (On-Site)
• 3 Months ago
Epic Games - Marketing Performance Manager

Epic Games

Cary, North Carolina, United States (On-Site)
• 1 Month ago
Meta - Global Sales Analytics Lead

Meta

Menlo Park, California, United States (Remote)
• 3 Months ago
Meta - Global Sales Analytics Lead

Meta

San Francisco, California, United States (Remote)
• 3 Months ago
N-iX - Data Analyst / ETL Engineer

N-iX

Ukraine (Remote)
• 5 Days ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Studio Wildcard - Engine Programmer

Studio Wildcard

Bellevue, Washington, United States (On-Site)
• 6 Months ago
Assystems - Lead Developer -  H/F

Assystems

Lyon, Auvergne-Rhône-Alpes, France (Hybrid)
• 3 Months ago
Nagarro - Senior QA Lead (all genders)

Nagarro

Austria (Remote)
• 4 Months ago
LeoVegas - Node.js Engineer

LeoVegas

Málaga, Andalusia, Spain (On-Site)
• 4 Months ago
PwC - ETIC, Cloud Solution Architect (Multi-Cloud, DevOps Focus) - Senior Manager

PwC

Cairo, Cairo Governorate, Egypt (On-Site)
• 3 Months ago
PlayStation Global - Staff Software Engineer

PlayStation Global

Madison, Wisconsin, United States (Hybrid)
• 3 Weeks ago
Nagarro - Senior Staff Engineer, .Net Fullstack

Nagarro

Hyderabad, Telangana, India (On-Site)
• 4 Months ago
ION - Cloud Engineer Kubernetes

ION

Collecchio, Emilia-Romagna, Italy (Hybrid)
• 4 Months ago
Next Level Business Services - Adobe CQ5/AEM Developer (Full Time)

Next Level Business Services

Sunnyvale, California, United States (On-Site)
• 3 Months ago
Nagarro - Staff Engineer, Java

Nagarro

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
• 3 Months ago

Get notifed when new similar jobs are uploaded

Jobs in New Delhi, Delhi, India

Arcadis - CADWorx P&ID Designer

Arcadis

Maharashtra, India (On-Site)
• 4 Months ago
CloudHire - SAP ABAP Workflow - Consultant

CloudHire

Gurugram, Haryana, India (Remote)
• 3 Months ago
Mettler-Toledo International,  Inc  - Software Engineer Test – Senior

Mettler-Toledo International, Inc

Karnataka, India (Hybrid)
• 4 Months ago
Paytm - Sales Team Lead - Fastag - Chennai

Paytm

Chennai, Tamil Nadu, India (On-Site)
• 2 Months ago
Advarra - Sr Software Engineer

Advarra

Bengaluru, Karnataka, India (Hybrid)
• 3 Months ago
RCP Hospitality   - Marketing Manager

RCP Hospitality

Kochi, Kerala, India (On-Site)
• 4 Months ago
totalitycorp - iOS Developer

totalitycorp

Gurugram, Haryana, India (On-Site)
• 1 Month ago
Assystems - Draftsman structure

Assystems

Mumbai, Maharashtra, India (On-Site)
• 3 Months ago
Rubrik - Senior Offensive Security Engineer - Product Security

Rubrik

Bengaluru, Karnataka, India (On-Site)
• 3 Months ago
STAGE - Product Manager - Monetization

STAGE

Noida, Uttar Pradesh, India (On-Site)
• 6 Months ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

The Walt Disney Company - Manager, Data Science

The Walt Disney Company

Burbank, California, United States (On-Site)
• 2 Weeks ago
Playtika - Experienced Business Analyst

Playtika

Israel (On-Site)
• 3 Months ago
Duolingo - Senior Data Scientist

Duolingo

Pittsburgh, Pennsylvania, United States (On-Site)
• 4 Months ago
WebMD - Data Analyst (m/w/d)

WebMD

United Kingdom (On-Site)
• 4 Months ago
Duolingo - Data Scientist, Intern (PhD or Masters)

Duolingo

New York, New York, United States (On-Site)
• 4 Months ago
My Fitness Pal - Data Analytics Engineer III

My Fitness Pal

United States (Remote)
• 1 Month ago
Stream Hatchet - Senior Data Engineer

Stream Hatchet

Barcelona, Catalonia, Spain (Hybrid)
• 4 Months ago
Brillio - Senior Data Specialist- R01531001

Brillio

Bengaluru, Karnataka, India (Hybrid)
• 4 Months ago
Epic Games - Data Analyst

Epic Games

Cary, North Carolina, United States (On-Site)
• 5 Days ago
The Walt Disney Company - Lead Data Engineer, Data Reliability

The Walt Disney Company

Santa Monica, California, United States (On-Site)
• 1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.

Gothenburg, Västra Götaland County, Sweden (On-Site)

New Delhi, Delhi, India (Remote)

Poland, Ohio, United States (Remote)

Kraków, Lesser Poland Voivodeship, Poland (On-Site)

Wrocław, Lower Silesian Voivodeship, Poland (On-Site)

Ukrainka, Kyiv Oblast, Ukraine (Remote)

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (On-Site)

Bengaluru, Karnataka, India (On-Site)

Mississauga, Ontario, Canada (On-Site)

View All Jobs

Get notified when new jobs are added by Luxoft

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug