Middle HPC Data Engineer

1 Week ago • 2 Years + • Data Analyst

Job Summary

Job Description

This role involves designing and implementing a high-performance data service module for an HPC modeling project. Responsibilities include developing parallel and concurrent I/O mechanisms using HDF5 for efficient data storage and retrieval of large-scale datasets. The engineer will optimize I/O operations for CPU/GPU workflows, implement caching and compression strategies, design data structures for 3D grid data, ensure data integrity, and conduct performance testing and benchmarking. The position requires strong programming skills (C++, Python, GoLang, or Fortran), expertise with HDF5 and parallel I/O frameworks (MPI I/O, POSIX I/O), and experience in optimizing I/O for large datasets. Collaboration with development and visualization teams is essential.
Must have:
  • HDF5 expertise for parallel I/O
  • C++, Python, GoLang, or Fortran proficiency
  • Experience with HPC data services
  • Parallel I/O framework knowledge (MPI, POSIX)
  • Performance optimization skills
Good to have:
  • Knowledge of Airflow, Prefect, or Dagster
  • Familiarity with scientific computing data formats
  • 3D visualization and simulation workflow experience
  • SQL and RDMS experience
Perks:
  • Flexible working format
  • Competitive salary and benefits
  • Personalized career growth
  • Professional development tools
  • Education reimbursement
  • Corporate events

Job Details

About our customer:

Our client is a global company specializing in software development and consulting who combines science and technology with deep industry expertise to solve complex subsurface and surface challenges in the evolving energy sector.
The client headquartered is located in Canada, with offices around the world. The company serves organizations globally, providing cutting-edge software technology and unparalleled customer support.

Key Software Solutions:
Our client offers a range of reservoir simulation software, including:

  • IMEX – A black oil simulator for primary, secondary, and tertiary recovery processes.
  • GEM – An advanced simulator for compositional, chemical, and unconventional reservoir modeling.
  • STARS – The industry standard for thermal and advanced recovery processes.
  • CMOST – An intelligent optimization and analysis tool that integrates statistical analysis, machine learning, and unbiased data interpretation to determine optimal reservoir solutions.


The company invests in research and development, continuously improving its products and delivering state-of-the-art solutions for energy modeling and optimization.

About the Role:

We are looking for an experienced Data Service Module Engineer to develop and deploy the data service module for the HPC modeling project. This role focuses on implementing high-performance data storage and retrieval systems using HDF5 or similar, with parallel and concurrent I/O capabilities. The ideal candidate will have expertise in designing scalable data services optimized for HPC or distributed workflows, ensuring low latency and high throughput.

 

Key Responsibilities:

  • Design and implement the data service module using HDF5 for efficient data storage and retrieval.
  • Develop parallel and concurrent I/O mechanisms to optimize performance for large-scale datasets.
  • Ensure the module is tightly integrated with HPC and visualization workflows.
  • Optimize I/O operations for CPU/GPU-based workflows to minimize bottlenecks.
  • Implement caching, compression, and other strategies to enhance performance.
  • Design data structures and schemas suitable for storing 3D grid data and other simulation outputs.
  • Ensure data integrity and consistency during concurrent read/write operations.
  • Develop and execute test cases to validate module performance and reliability under various load conditions.
  • Conduct benchmarking to ensure scalability across different hardware configurations.
  • Document the architecture, APIs, and usage guidelines for the data service module.
  • Provide technical support to the development and visualization teams for data integration.

 

Requirements:

  • Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related fields.
  • 2+ years of experience in developing and deploying data services for HPC or similar systems.
  • Proven expertise with HDF5 or similar, in parallel I/O operations. Equivalent experience in distributed systems is also applicable.
  • Programming: Strong proficiency in (at least one): C++, Python, GoLang, or Fortran.
  • HDF5 Expertise: In-depth knowledge of HDF5 APIs and advanced features like parallel HDF5.
  • Parallel I/O: Experience with MPI I/O, POSIX I/O, or similar frameworks for concurrent/parallel data access.
  • Performance Optimization: Skills in profiling and optimizing I/O operations for large datasets.
  • Proficiency in SQL and experience with any RDMS
  • Might be a plus: knowledge of at least one orchestration and scheduling tool, for example, Airflow, Prefect, Dagster, etc.
  • Strong problem-solving skills and ability to work in a multidisciplinary team.
  • Excellent communication skills for cross-team collaboration and documentation.

 

Preferred Qualifications:

  • Familiarity with data formats used in scientific computing, 3D visualization, and simulation workflows.

We offer:

  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits

Similar Jobs

ByteDance - Machine Learning Engineer, SMB

ByteDance

San Jose, California, United States (On-Site)
6 Days ago
Hawk Eye Innovations - Computer Vision Engineer - Level 2

Hawk Eye Innovations

Budapest, Hungary (Hybrid)
6 Days ago
Toptracer - Embedded Software Engineer

Toptracer

Stockholm, Stockholm County, Sweden (Hybrid)
2 Months ago
Backbone - Electrical Engineer

Backbone

Atherton, California, United States (Hybrid)
8 Months ago
Fluence - Sr. Quality Assurance Engineer II

Fluence

Houston, Texas, United States (Hybrid)
5 Months ago
Fortis Games - Head of Analytics

Fortis Games

United States (On-Site)
2 Months ago
ION - Data Associate - KYC6

ION

Budapest, Hungary (On-Site)
5 Months ago
Razer - Senior Data Scientist

Razer

Kuala Lumpur, Federal Territory Of Kuala Lumpur, Malaysia (On-Site)
6 Months ago
Next Level Business Services - Markit EDM

Next Level Business Services

Pittsburgh, Pennsylvania, United States (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

G5 Games - C++ Gameplay Programmer

G5 Games

Limassol, Limassol, Cyprus (Remote)
5 Months ago
The Walt Disney Company - Director, Machine Learning Engineering

The Walt Disney Company

California, United States (On-Site)
1 Week ago
Scorewarrior - Backend C++ Developer

Scorewarrior

Limassol, Limassol, Cyprus (On-Site)
10 Months ago
ByteDance - Backend Engineer Intern, Video-On-Demand

ByteDance

Singapore (On-Site)
1 Week ago
Scorewarrior - Core Client Senior C++ Developer

Scorewarrior

Limassol, Limassol, Cyprus (On-Site)
6 Days ago
Haptic - Lead Engine Software Engineer

Haptic

Dallas, Texas, United States (Remote)
2 Months ago
Probably Monsters - Systems Engineer, Engine (Core Technology)

Probably Monsters

Texas, United States (On-Site)
1 Month ago
Kojima Productions - System Programmer

Kojima Productions

Tokyo, Japan (On-Site)
4 Months ago
Advitha Tech Solutions - C++/Multimedia Engineer

Advitha Tech Solutions

New Delhi, Delhi, India (Remote)
6 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Argentina

Devoted Studios - Social Media Manager

Devoted Studios

Argentina (Remote)
2 Weeks ago
Red Point Labs - Unity Game Developer (Remote OK)

Red Point Labs

Argentina (Remote)
10 Months ago
Evolution - Game Presenter

Evolution

Buenos Aires, Buenos Aires, Argentina (On-Site)
8 Months ago
N-iX - Senior QA Engineer

N-iX

Argentina (Remote)
1 Week ago
Matific - Product Support Engineer

Matific

Buenos Aires, Buenos Aires, Argentina (Hybrid)
1 Week ago
Evolution - Card Inspector

Evolution

Buenos Aires, Buenos Aires, Argentina (On-Site)
10 Months ago
N-iX - Senior .NET Desktop Engineer

N-iX

Argentina (Remote)
1 Week ago
Evolution - Technical Integration Specialist

Evolution

Buenos Aires, Buenos Aires, Argentina (On-Site)
6 Months ago
Go Fund Me - Senior Software Engineer (Backend)

Go Fund Me

Buenos Aires, Buenos Aires, Argentina (Hybrid)
2 Months ago
Matific - Product Support Agent

Matific

Buenos Aires, Buenos Aires, Argentina (Hybrid)
1 Week ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

ASSIST Software - Data Engineer

ASSIST Software

Suceava, Suceava County, Romania (Remote)
1 Month ago
The Walt Disney Company - Sr Workforce Analyst

The Walt Disney Company

Burbank, California, United States (On-Site)
1 Week ago
seeking alpha - Senior Data Scientist

seeking alpha

Ukraine (Remote)
2 Months ago
Evolution - Data Scientist

Evolution

Warsaw, Masovian Voivodeship, Poland (Hybrid)
8 Months ago
PlayStation Global - Manager, Analytics Engineering

PlayStation Global

Aliso Viejo, California, United States (On-Site)
6 Days ago
Entain - Risk Analyst

Entain

Brisbane, Queensland, Australia (Hybrid)
6 Days ago
Ubisoft - Esports Business Intelligence Specialist

Ubisoft

Paris, Île-de-France, France (Hybrid)
2 Weeks ago
Playrix - Senior Data Analyst (Attribution)

Playrix

Ireland (Remote)
5 Months ago
Trend Micro - (Sr.) Data Engineer/AI Trainer

Trend Micro

Taipei City, Taiwan (On-Site)
6 Months ago
Morning Star - ESG Research Senior Analyst

Morning Star

Bucharest, Bucharest, Romania (Hybrid)
6 Months ago

Get notifed when new similar jobs are uploaded