Senior Data Engineer

5 Months ago • 3 Years + • Data Analysis

Job Summary

Job Description

As a Senior Data Engineer at BestEx Research, you will be involved in designing and managing execution algorithms for global equities, futures, and foreign exchange. Your responsibilities will include designing and implementing ETL pipelines, managing data infrastructure, designing scalable data systems, developing analytics and reporting tools, and working with large financial datasets. You will collaborate with Quants and Product teams, and document methodologies and system architecture. This role requires strong problem-solving skills, experience with data orchestration tools, system design, and database management.
Must have:
  • Design and implement robust ETL pipelines for financial data.
  • Design and maintain databases and automate daily tasks.
  • Architect scalable and efficient data systems.
  • Develop and optimize production-level code for reports.
  • Work with large financial data sets, applying data wrangling techniques.
  • Document methodologies, system architecture, and data flows.
  • Collaborate with Quants and Product teams.
Good to have:
  • Experience with building API backends.
Perks:
  • Modern work spaces.
  • Free parking.
  • Recreational games.
  • Wellness room.
  • Free meals.

Job Details

About BestEx Research

BestEx Research is a financial technology and research firm specializing in building sophisticated execution algorithms and transaction cost modeling tools servicing multiple asset classes. The firm provides its services to performance-demanding hedge funds, CTAs, asset managers, and banks through a traditional electronic broker and in a broker-neutral Software as a Service (SaaS) model.

BestEx Research's mission is to become the leader in automation and measurement of execution across asset classes globally and significantly reduce transaction costs for our clients.

Its cloud-based platform, Algo Management System (AMS), is the first end-to-end algorithmic trading solution for equities, futures, and foreign exchange that delivers an entire ecosystem around execution algorithms, including transaction cost analysis (TCA), an algo customization tool called Strategy Studio, a trading dashboard, and pre-trade analytics in a single platform. The platform is currently live for US equity and global futures trading.

BestEx Research is disrupting a $100 billion industry by challenging the status quo of stale, black-box solutions from banks and offering next-generation execution algorithms that combine performance improvement with transparency and customization. BestEx Research uses leading-edge technology to support its low-latency, highly scalable research and trading systems with its back end in C++, research libraries in C++/Python and R, and web-based technologies for delivering its front-end platforms.

Visit bestexresearch.com for more information about our mission, products, research, and services.

Why work at BestEx Research?

If you want to help investors reduce their trading costs in an incredibly complex market structure and help create and shape an industry-defining product, then this is an excellent opportunity to join an incredibly talented team of 50+. BestEx Research has almost zero turnover, top notch clients, zero bureaucracy, and a flat organizational structure. It is a true meritocracy, set in a collaborative environment, where every employee is working on extraordinarily interesting projects. Our pay scale and benefits are comparable to top-tier firms in our industry, either in the form of cash compensation or a hybrid cash and equity compensation plan. Employees at BestEx Research have exposure to much more variety in the projects they complete and opportunities for growth than in similar roles at other firms.

Location: Bengaluru, India

Our beautiful Bangalore office is conveniently located in Outer Ring Rd, Bangalore. Amenities include modern work spaces, free parking, recreational games, wellness room, and free meals.

Primary Job Responsibilities:

Senior Data Engineer in the Data Engineering team contributes to the design and management of execution algorithms for global equities, futures, and foreign exchange with the aim of continuously minimizing trading costs for our clients. BestEx Research data engineers will learn about algorithmic trading, data management, performance measurement, and market microstructure, globally. The data engineering team is nested within a talented team of experts and practitioners in the field. Our internal research environment includes tick-by-tick market data, historical back testing simulators, instrument-level daily analytics and normalized order and execution data to facilitate new research. Day-to-day responsibilities of this role may include but are not limited to:

  • ETL Pipeline Design & Implementation - Design, implement and optimize robust ETL pipelines for processing large volumes of financial market data, including tick-by-tick data, ensuring data quality and integrity.
  • Data Infrastructure & Automation - Design and maintain databases (schemas, design, functions) and automate daily BAU tasks, ensuring smooth data workflows and managing data dependencies effectively. Investigate issues and implement solutions to maintain data integrity.
  • System Design - Architect scalable and efficient data systems to handle high-volume, high-frequency financial data, with a focus on performance optimization and reliability.
  • Analytics and Reporting Tools - Develop and optimize production-level code for new reports and analyses, ensuring they integrate seamlessly into both the post-trade product and pre-trade analytics tools.
  • Data Management & Processing - Work with large financial data sets, applying efficient data wrangling techniques to ensure clean, structured, and ready-to-use data for analysis. Investigate issues and perform custom analysis for special market events or days, ensuring the highest level of accuracy and performance.
  • Documentation - Document methodologies, system architecture, and data flows clearly for a range of internal and external audiences, ensuring transparency and reproducibility.
  • Cross-Collaboration - Work closely with Quants and Product teams, contributing in data collection, model development, evaluation, and continuous improvement.

  • Highly motivated, willing to take ownership of work
  • Collaborative mindset with strong independent problem-solving abilities
  • Minimum 3+ years of experience in designing and implementing ETL pipelines
  • Strong experience with data orchestration tools such as Airflow or Dagster
  • Proven experience in system design for large-scale data processing
  • Experience maintaining and managing large databases and raw tick-by-tick data
  • Experience with Bitbucket, Git, Confluence, and Jira
  • Bachelor's degree in Engineering or Master's degree in computer science, data science, statistics, or other quantitative fields
  • Strong command of PostgreSQL and proficiency in Python or R
  • Experience with building API backends is valued but not required
  • Excellent communication skills and ability to articulate complex technical concepts to colleagues and clients

Similar Jobs

Lead Venture - AR Manager

Lead Venture

Guadalajara, Jalisco, Mexico (Remote)
1 Week ago
Highspot - Sr. Data Analyst

Highspot

Hyderabad, Telangana, India (Hybrid)
5 Months ago
Zuora - Product Manager, Billing

Zuora

Chennai, Tamil Nadu, India (Remote)
1 Week ago
Sprinkler - Senior Technical Support Engineer - CCaaS

Sprinkler

Bengaluru, Karnataka, India (On-Site)
2 Months ago
N-ix - Senior Data Science/ AI Engineer (with Databricks)

N-ix

Poland (On-Site)
1 Month ago
Hawkeye Innovations - Senior Data Test Automation Engineer

Hawkeye Innovations

Basingstoke, England, United Kingdom (Hybrid)
2 Weeks ago
miniclip - Senior Data Analyst - 12 Month Fix-term

miniclip

London, England, United Kingdom (On-Site)
1 Month ago
Lightcast - Data Scientist

Lightcast

United States (Remote)
1 Month ago
CrazyLabs - Data Engineer

CrazyLabs

Skopje, Greater Skopje, North Macedonia (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Enverus - Business Development Representative, South-East Asia

Enverus

Singapore (On-Site)
2 Weeks ago
Aera Technology - Client Partner

Aera Technology

Boston, Massachusetts, United States (Hybrid)
5 Months ago
Safe security - Enterprise Account Executive

Safe security

Philadelphia, Pennsylvania, United States (Remote)
9 Months ago
Lytx,  Inc  - Partner Development Manager, OEM

Lytx, Inc

United States (Remote)
1 Month ago
high radius - Consultant

high radius

Hyderabad, Telangana, India (On-Site)
1 Week ago
Vimeo - Vice President, Sales & Account Management, EMEA

Vimeo

London, England, United Kingdom (On-Site)
4 Weeks ago
Mixpanel - Account Manager, Mid-Market

Mixpanel

San Francisco, California, United States (Hybrid)
2 Weeks ago
deel. - Account Executive, SMB | ASEAN

deel.

Singapore (Remote)
1 Week ago
Everlaw - Manager, Product Lead

Everlaw

Oakland, California, United States (Hybrid)
2 Weeks ago
Prophecy - Senior Partner Manager, Partnerships

Prophecy

United States (Remote)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Bengaluru, Karnataka, India

hogarth - AI Business Analyst

hogarth

Chennai, Tamil Nadu, India (On-Site)
1 Week ago
HP - Software Applications Engineer 5

HP

Bengaluru, Karnataka, India (On-Site)
1 Week ago
Accenture - Test Automation Lead

Accenture

Bengaluru, Karnataka, India (On-Site)
3 Months ago
DevRev - Customer Success Manager

DevRev

Chennai, Tamil Nadu, India (On-Site)
2 Months ago
PhonePe - Server Administrator (Patching)

PhonePe

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Capgemini - DevOps Senior Consultant / Manager

Capgemini

Pune, Maharashtra, India (Hybrid)
1 Month ago
Dentsu - Senior Account Manager - Data Science

Dentsu

Chennai, Tamil Nadu, India (On-Site)
1 Month ago
Hitachi - F&O Finance Functional_NA MS

Hitachi

Hyderabad, Telangana, India (On-Site)
9 Months ago
Nagarro - Senior Staff Consultant ,SAP Analytics Data Manageme

Nagarro

India (Remote)
9 Months ago
Fictiv - Corporate Presentation Designer

Fictiv

Pune, Maharashtra, India (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Roblox - Data Scientist

Roblox

Gurugram, India (On-Site)
1 Month ago
Monzo - Lead Data Scientist

Monzo

London, England, United Kingdom (Remote)
2 Months ago
Sword Health - Senior Data Analyst

Sword Health

Porto, Porto District, Portugal (Hybrid)
2 Months ago
Saronic Technologies - Program Data Analyst

Saronic Technologies

Austin, Texas, United States (On-Site)
1 Week ago
Pomelo - Senior Software Engineer, Core Data

Pomelo

United States (Remote)
2 Weeks ago
DraftKings - Senior Data Engineer, Data Enablement

DraftKings

Boston, Massachusetts, United States (On-Site)
4 Months ago
quience - Senior Data Analyst

quience

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Minecast - Senior Data Scientist

Minecast

Lexington, Massachusetts, United States (On-Site)
1 Week ago
Tide - Lead Data Scientist

Tide

Hyderabad, Telangana, India (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded