Middle HPC Data Engineer

2 Months ago • 2 Years + • Data Analyst

Job Summary

Job Description

This role involves designing and implementing a high-performance data service module for an HPC modeling project. Responsibilities include developing parallel and concurrent I/O mechanisms using HDF5 for efficient data storage and retrieval of large-scale datasets. The engineer will optimize I/O operations for CPU/GPU workflows, implement caching and compression strategies, design data structures for 3D grid data, ensure data integrity, and conduct performance testing and benchmarking. The position requires strong programming skills (C++, Python, GoLang, or Fortran), expertise with HDF5 and parallel I/O frameworks (MPI I/O, POSIX I/O), and experience in optimizing I/O for large datasets. Collaboration with development and visualization teams is essential.
Must have:
  • HDF5 expertise for parallel I/O
  • C++, Python, GoLang, or Fortran proficiency
  • Experience with HPC data services
  • Parallel I/O framework knowledge (MPI, POSIX)
  • Performance optimization skills
Good to have:
  • Knowledge of Airflow, Prefect, or Dagster
  • Familiarity with scientific computing data formats
  • 3D visualization and simulation workflow experience
  • SQL and RDMS experience
Perks:
  • Flexible working format
  • Competitive salary and benefits
  • Personalized career growth
  • Professional development tools
  • Education reimbursement
  • Corporate events

Job Details

About our customer:

Our client is a global company specializing in software development and consulting who combines science and technology with deep industry expertise to solve complex subsurface and surface challenges in the evolving energy sector.
The client headquartered is located in Canada, with offices around the world. The company serves organizations globally, providing cutting-edge software technology and unparalleled customer support.

Key Software Solutions:
Our client offers a range of reservoir simulation software, including:

  • IMEX – A black oil simulator for primary, secondary, and tertiary recovery processes.
  • GEM – An advanced simulator for compositional, chemical, and unconventional reservoir modeling.
  • STARS – The industry standard for thermal and advanced recovery processes.
  • CMOST – An intelligent optimization and analysis tool that integrates statistical analysis, machine learning, and unbiased data interpretation to determine optimal reservoir solutions.


The company invests in research and development, continuously improving its products and delivering state-of-the-art solutions for energy modeling and optimization.

About the Role:

We are looking for an experienced Data Service Module Engineer to develop and deploy the data service module for the HPC modeling project. This role focuses on implementing high-performance data storage and retrieval systems using HDF5 or similar, with parallel and concurrent I/O capabilities. The ideal candidate will have expertise in designing scalable data services optimized for HPC or distributed workflows, ensuring low latency and high throughput.

 

Key Responsibilities:

  • Design and implement the data service module using HDF5 for efficient data storage and retrieval.
  • Develop parallel and concurrent I/O mechanisms to optimize performance for large-scale datasets.
  • Ensure the module is tightly integrated with HPC and visualization workflows.
  • Optimize I/O operations for CPU/GPU-based workflows to minimize bottlenecks.
  • Implement caching, compression, and other strategies to enhance performance.
  • Design data structures and schemas suitable for storing 3D grid data and other simulation outputs.
  • Ensure data integrity and consistency during concurrent read/write operations.
  • Develop and execute test cases to validate module performance and reliability under various load conditions.
  • Conduct benchmarking to ensure scalability across different hardware configurations.
  • Document the architecture, APIs, and usage guidelines for the data service module.
  • Provide technical support to the development and visualization teams for data integration.

 

Requirements:

  • Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related fields.
  • 2+ years of experience in developing and deploying data services for HPC or similar systems.
  • Proven expertise with HDF5 or similar, in parallel I/O operations. Equivalent experience in distributed systems is also applicable.
  • Programming: Strong proficiency in (at least one): C++, Python, GoLang, or Fortran.
  • HDF5 Expertise: In-depth knowledge of HDF5 APIs and advanced features like parallel HDF5.
  • Parallel I/O: Experience with MPI I/O, POSIX I/O, or similar frameworks for concurrent/parallel data access.
  • Performance Optimization: Skills in profiling and optimizing I/O operations for large datasets.
  • Proficiency in SQL and experience with any RDMS
  • Might be a plus: knowledge of at least one orchestration and scheduling tool, for example, Airflow, Prefect, Dagster, etc.
  • Strong problem-solving skills and ability to work in a multidisciplinary team.
  • Excellent communication skills for cross-team collaboration and documentation.

 

Preferred Qualifications:

  • Familiarity with data formats used in scientific computing, 3D visualization, and simulation workflows.

We offer:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

Similar Jobs

Rockstar Games - Associate Animator: Gameplay

Rockstar Games

Oakville, Ontario, Canada (On-Site)
2 Months ago
Adobe - Support Service Manager

Adobe

Saudi Arabia (On-Site)
4 Months ago
Cavnue - Senior Integration Engineer

Cavnue

(Remote)
4 Weeks ago
Google - Senior Software Engineering, iOS

Google

Durham, North Carolina, United States (On-Site)
1 Month ago
Adobe - Machine Learning Engineer

Adobe

San Jose, California, United States (On-Site)
1 Month ago
PENN Interactive - Senior Manager, Product Intelligence, AI/ML & Data Solutions

PENN Interactive

Philadelphia, Pennsylvania, United States (Hybrid)
2 Months ago
Kwalee - Product Analyst

Kwalee

Royal Leamington Spa, England, United Kingdom (On-Site)
4 Months ago
SmileGate - AI Reinforcement Learning and Predictive Model Development

SmileGate

Seongnam-si, Gyeonggi-do, South Korea (On-Site)
2 Months ago
Super - Manager, Data Analytics

Super

Canada (Remote)
3 Months ago
Hike - Senior Analytics Manager

Hike

Delhi, India (Remote)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Aryaka - UI_UX Engineer

Aryaka

Bengaluru, Karnataka, India (On-Site)
1 Year ago
Gameplay Galaxy - Senior Game Programmer

Gameplay Galaxy

British Virgin Islands (Remote)
4 Months ago
Google - Senior Software Engineer, SDLC, Gemini Code Assist

Google

Sunnyvale, California, United States (On-Site)
1 Month ago
Acceldata - Senior Platform Engineer

Acceldata

Bengaluru, Karnataka, India (On-Site)
8 Months ago
Google - Senior Software Engineer, Recommendations, Ranking and Prediction, Google Workspace

Google

Sunnyvale, California, United States (On-Site)
1 Month ago
Visual Concepts - Engineer

Visual Concepts

Austin, Texas, United States (On-Site)
2 Months ago
Sperasoft - Senior 3D Vegetation Artist

Sperasoft

Yerevan, Yerevan, Armenia (On-Site)
2 Months ago
Google - Software Engineer III, Security Endpoint Agents, Core

Google

Dublin, County Dublin, Ireland (On-Site)
1 Month ago
Ubisoft - Senior Programmer [Unity]

Ubisoft

Shanghai, Shanghai, China (On-Site)
8 Months ago
Google - Senior Software Engineer, Infrastructure, Google Cloud

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in Argentina

Betson Group - HR Generalist

Betson Group

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Month ago
Evolution - HR Generalist JR

Evolution

Ezeiza, Buenos Aires Province, Argentina (On-Site)
6 Months ago
Devrev - Customer Success Manager

Devrev

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Month ago
Devrev - Revenue: Sales Development Representative

Devrev

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Month ago
Evolution - Service Support Specialist (On-Site Canning, Ezeiza)

Evolution

Buenos Aires, Buenos Aires, Argentina (On-Site)
6 Months ago
PwC - Senior - Process Audit, Governance, Risk, and Compliance (SOX)

PwC

Buenos Aires, Buenos Aires, Argentina (Hybrid)
2 Months ago
The Walt Disney Company - Analyst, Marketing Insights

The Walt Disney Company

Buenos Aires, Buenos Aires, Argentina (On-Site)
1 Month ago
Dialpad AI - Senior QA Engineer

Dialpad AI

Buenos Aires, Buenos Aires, Argentina (On-Site)
3 Weeks ago
Room 8 Studio - Realistic Environment Concept Artist, Senior

Room 8 Studio

Argentina (On-Site)
4 Months ago
N-iX - Senior HPC Data Engineer

N-iX

Argentina (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

ION - Internship - Data Science

ION

Milan, Lombardy, Italy (On-Site)
7 Months ago
Match Group - Sr. Software Engineer, Data

Match Group

Palo Alto, California, United States (Hybrid)
7 Months ago
Amazon Games - Senior Business Analyst, Luna Data Insights and Intelligence

Amazon Games

Bellevue, Washington, United States (On-Site)
1 Month ago
N-iX - Senior Manual QA (With Data Experience)

N-iX

Ukraine (Remote)
1 Month ago
Voodoo - Senior Data Analyst

Voodoo

Paris, Île-de-France, France (Hybrid)
2 Months ago
Google - Scaled Abuse Analyst, YouTube Trust and Safety

Google

Hyderabad, Telangana, India (On-Site)
1 Month ago
Netflix - Senior Data Engineer, Commerce Product

Netflix

Mexico City, Mexico City, Mexico (On-Site)
3 Months ago
Coinbase - Senior Data Scientist

Coinbase

Hyderabad, Telangana, India (Remote)
8 Months ago
Playtech - Data Engineer

Playtech

Cyprus (On-Site)
2 Months ago
Hawk Eye Innovations - Senior Data Test Automation Engineer

Hawk Eye Innovations

Basingstoke, England, United Kingdom (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded