Data Engineer

6 Days ago • 3-5 Years

About the job

Summary_{By Outscal}

Data Engineer role at ION Analytics focused on building and managing data pipelines for production ML models. Requires 3+ years experience in data engineering, Databricks, PySpark, Apache Spark, and strong Python skills. Experience with cloud infrastructure (AWS/Azure), data warehousing, and data lakes is essential.

The Role:

As a Data Science - Data Engineer, you will play a crucial role in creating and managing the dataflows used by production machine learning models and data scientists. You will be responsible for collaborating with cross-functional teams, engineers, product managers and data scientists to understand requirements and deliver high-quality solutions. You will be expected to implement best practices in data management, coding, testing, and deployment, ensuring scalability, performance, and security. Strong problem-solving skills, excellent communication, and the ability to adapt to new challenges in a fast-paced environment is essential for this role.

Key Responsibilities

Design, build, and maintain large-scale data pipelines to ingest, process, and transform complex financial data
Develop data ingestion frameworks to handle high-volume and high-variety data feeds
Implement data quality control measures to detect and handle data anomalies, duplicates, and inconsistencies
Optimize data pipeline performance for scalability, reliability, and latency
Collaborate with data scientists to integrate machine learning models into the data pipeline and ensure seamless data flow
Develop and maintain model serving infrastructure to support real-time predictions and batch scoring
Implement model monitoring and logging to track performance, data quality, and model drift
Manage cloud-based infrastructure to support data pipeline and model serving workloads
Troubleshoot data pipeline issues, data quality problems, and model performance degradation
Collaborate with cross-functional teams, including data science, product, and engineering teams to ensure alignment and effective communication

Required Skills, Experience, and Qualifications

3+ years of experience in a data engineering or similar role, with a focus on building and deploying data pipelines and machine learning models
2+ years of experience with Databricks, PySpark, and Apache Spark
1+ year of experience with Apache Airflow, Apache Kafka, and Kubernetes
Strong Python skills, with 3+ years of experience
Experience with production machine learning model deployment and management
Knowledge of cloud-based infrastructure and services AWS and Azure
Experience with data warehousing, data lakes, and data marts
Familiarity with containerization using Docker and orchestration using Kubernetes
Understanding of messaging and streaming using Apache Kafka
Experience with workflow management using Apache Airflow
Strong problem-solving skills, with the ability to troubleshoot complex data pipeline and machine learning model issues
Excellent communication and collaboration skills, with the ability to work with data scientists, engineers, and senior stakeholders
Degree educated in Computer Science or relevant subject
Excellent spoken and written English
Experience of driving innovation within teams
Positive, can-do approach to work, delivering on commitments
Able to earn respect and build trust through high-quality delivery and by demonstrating value added
Act with integrity, with the ability to make decisions, taking accountability for the consequences of these decisions
Creative approach to problem solving, can look at existing situations and problems in novel ways and propose innovative solutions

About us

We’re a diverse group of visionary innovators who provide trading and workflow automation software, high-value analytics, and strategic consulting to corporations, central banks, financial institutions, and governments. Founded in 1999, we’ve achieved tremendous growth by bringing together some of the best and most successful financial technology companies in the world.

• Over 2,000 of the world’s leading corporations, including 50% of the Fortune 500 and 30% of the world’s central banks, trust ION solutions to manage their cash, in-house banking, commodity supply chain, trading and risk.

• Over 800 of the world’s leading banks and broker-dealers use our electronic trading platforms to operate the world’s financial market infrastructure.

ION is a rapidly expanding and dynamic group with 13,000 employees and offices in more than 40 cities around the globe. Our ever-expanding global footprint, cutting edge products, and over 40,000 customers worldwide provide an unparalleled career experience for those who share our vision.

ION is committed to maintaining a supportive and inclusive environment for people with diverse backgrounds and experiences. We respect the varied identities, abilities, cultures, and traditions of the individuals who comprise our organization and recognize the value that different backgrounds and points of view bring to our business.

ION adheres to an equal employment opportunity policy that prohibits discriminatory practices or harassment against applicants or employees based on any legally impermissible factor.

15 skills required for this role

Boost these skills to join the top 1% applicants for this job

About The Company

Ion

685 Active Jobs

We’re visionary innovators who are delivering mission-critical trading and workflow automation software to financial institutions, corporations, central banks, and governments. By combining our passion for automation with a strategic view on the industries we serve, we design solutions that improve decision-making, simplify complex processes, and empower people. Simply put, we help our customers do more, faster and better than before. We believe our investments in research and development are shaping the future of automation and enabling our customers to transform their business. And we embrace the power of community, working with each other and with our customers to succeed through a positive culture of continuous improvement.