Outscal Logooutscal logo

Senior C++ Software Engineer - Apache Spark Solution

1 Month ago • 8 Years + • Data Analyst

Job Summary

Job Description

NVIDIA's Spark Acceleration group seeks experienced C++ Software Engineers to accelerate Apache Spark for data science. Responsibilities include designing and implementing a native Spark execution engine using RAPIDS, Velox, and UCX; optimizing data exchange between Velox and RAPIDS; enhancing Velox for performance and Spark compatibility; contributing to the RAPIDS library; conducting performance benchmarking; and presenting technical solutions. The ideal candidate will have 8+ years of software development experience, 3+ years with Velox, RAPIDS, or similar frameworks, exceptional C++ skills, and expertise in columnar data processing. Experience with GPU-accelerated libraries is a plus.
Must have:
  • 8+ years software development experience
  • 3+ years with Velox/RAPIDS
  • Exceptional C++ skills
  • Columnar data processing expertise
  • High-quality distributed system design
Good to have:
  • GPU-accelerated library experience (CUDA, cuBLAS, NCCL, RAPIDS, UCX)
  • Apache Spark, Hadoop, Hive, Arrow committership
Perks:
  • Highly competitive salary
  • Comprehensive benefits package

Job Details

We are seeking experienced C++ Software Engineers to join our Spark Acceleration group. Data scientists spend a considerable amount of time exploring data, iterating over machine learning (ML) experiments.  NVIDIA  believes that data science workflows can benefit tremendously from being accelerated, to enable data scientists to explore many more and larger datasets to drive towards their business goals, faster, and more reliably.

You will work with the open source community to accelerate Apache Spark for data science. Apache Spark is the most popular data processing engine in data centers for data science. We aim to dramatically accelerate Apache Spark use cases without application code changes. You will work on open source libraries including Spark-RAPIDS (https://github.com/NVIDIA/spark-rapids), RAPIDS (https://github.com/rapidsai), and Velox (https://github.com/facebookincubator/velox).

What you'll be doing:

  • Design and implement native Spark execution engine using RAPIDS, Velox, UCX and other related libraries. 

  • Design and implement solutions to optimize data exchange between Velox and RAPIDS libraries

  • Enhance Velox OSS library for improved performance and Spark compatibility

  • Contribute to RAPIDS library for large-scale adoptions in major enterprises

  • Conduct performance benchmarking and profiling to achieve speed-of-light performance

  • Working with a team of exceptional engineers including PMC and Committers of Apache Spark, Apache Hadoop, Apache Hive, and Apache Arrow

  • Presenting technical solutions in industry conferences and meetups

What we need to see:

  • BS, MS, or PhD in Computer Science, Computer Engineering, or closely related field 

  • 8+ years of work or research experience in software development

  • 3+ years hands-on development experience with Velox, RAPIDS or similar data processing frameworks in memory management techniques and data serialization

  • Exceptional C++ development experience in design, programming, testing, and debugging  

  • Design and development expertises in columnar data processing with SIMD (Single Instruction, Multiple Data) and vectorization techniques

  • Familiarity with operating systems and software development environments for ARM

  • Proven technical skills in designing and implementing high-quality distributed systems 

  • Able to work successfully with multi-functional teams across organizational boundaries and geographies

  • Highly motivated with strong communication skills

Ways to stand out from the crowd:

  • Committership at major open source big-data projects 

  • Working experience with GPU-accelerated libraries (CUDA, cuBLAS, NCCL, RAPIDS, UCX) 

We are an AA/EEO/Disabled employer and with highly competitive salaries and a comprehensive benefits package, NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most brilliant and talented people on the planet working for us. Are you creative and autonomous? Do you love a challenge? If so, we want to hear from you.

Similar Jobs

Playrix - Senior Game UI Developer

Playrix

Almaty, Almaty Region, Kazakhstan (Remote)
5 Months ago
Infinity Ward - Expert Animation Engineer

Infinity Ward

California, United States (On-Site)
1 Day ago
Creative Assembly - Mid/Senior Gameplay Programmer - CA Sofia

Creative Assembly

Sofia, Sofia City Province, Bulgaria (On-Site)
3 Months ago
Aesir Interactive - Unreal Engine Programmer Games (Regular/ Senior) (f/m/d)

Aesir Interactive

Munich, Bavaria, Germany (Hybrid)
7 Months ago
Hero Gaming - Business Intelligence Developer

Hero Gaming

Swieqi, Malta (On-Site)
1 Month ago
GoMotive - Senior Applied Scientist

GoMotive

India (Remote)
1 Week ago
PwC - IN_Senior Associate_Qlikview_D&A_Advisory _Bangalore

PwC

Bengaluru, Karnataka, India (On-Site)
5 Months ago
Trend Micro - (Sr.) Data Engineer/AI Trainer

Trend Micro

Taipei City, Taiwan (On-Site)
5 Months ago
PlayStation Global - Manager, Analytics Engineering

PlayStation Global

Aliso Viejo, California, United States (On-Site)
11 Hours ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Supercell - Senior Gameplay Programmer, Squad Busters

Supercell

Helsinki, Uusimaa, Finland (On-Site)
5 Months ago
Luxoft - Regular BSP Developer

Luxoft

Bengaluru, Karnataka, India (Hybrid)
4 Months ago
Passive Logic - AI Control Theory & Optimization Scientist

Passive Logic

Salt Lake City, Utah, United States (On-Site)
3 Months ago
ByteDance - Backend Engineer, ARK Large Model Platform (Singapore)

ByteDance

Singapore (On-Site)
4 Months ago
ByteDance - Lead Research Scientist, Foundation Model, Music Intelligence

ByteDance

San Jose, California, United States (On-Site)
4 Months ago
Ubisoft - Senior Animation Programmer [Snowdrop]

Ubisoft

Malmö, Skåne County, Sweden (On-Site)
1 Week ago
NVIDIA - Manager, Software Engineering

NVIDIA

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
2 Months ago
NVIDIA - Senior Verification Engineer - Hardware

NVIDIA

Canada (On-Site)
1 Month ago
Blizzard Entertainment - Senior Software Engineer, Server - Hearthstone | Irvine, CA

Blizzard Entertainment

Irvine, California, United States (Hybrid)
5 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Beijing, Beijing, China

NVIDIA - Senior System Software Engineer - Autonomous Driving

NVIDIA

Beijing, Beijing, China (On-Site)
1 Month ago
AppLovin - Analyst, Business Development

AppLovin

Beijing, Beijing, China (On-Site)
3 Months ago
Tencent - Senior Technical Artist UE5

Tencent

Shenzhen, Guangdong Province, China (On-Site)
3 Months ago
Tencent - Senior Technical Artist (VFX Rendering) - Global Realistic 3A Action Game

Tencent

Shenzhen, Guangdong Province, China (On-Site)
3 Months ago
Tencent - Senior 3D Character Designer (AAA Stylized Realistic Shooting Game)

Tencent

Shenzhen, Guangdong Province, China (On-Site)
2 Months ago
Riot Games - Game Designer, Level-Design - R&D

Riot Games

Shanghai, Shanghai, China (On-Site)
1 Day ago
NVIDIA - Senior Technical Program Manager, AI Datacenter

NVIDIA

Shenzhen, Guangdong Province, China (On-Site)
1 Month ago
NVIDIA - Manufacturing Engineer

NVIDIA

Shanghai, Shanghai, China (On-Site)
3 Weeks ago
Riot Games - Senior Game Product Manager (Monetization and Growth) - FFGS

Riot Games

Shanghai, Shanghai, China (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

PwC - Associate Data and Analytics Consultant

PwC

Delhi, India (On-Site)
2 Months ago
The Walt Disney Company - Graduate Associate Research Analyst, People Insights

The Walt Disney Company

Lake Buena Vista, Florida, United States (On-Site)
1 Day ago
Crunchyroll - Senior Investigations Analyst

Crunchyroll

Dallas, Texas, United States (On-Site)
2 Weeks ago
Animoca Brands - Quant Researcher

Animoca Brands

Hong Kong (On-Site)
6 Months ago
GoTo Group - Senior Data Scientist (India)

GoTo Group

Bengaluru, Karnataka, India (On-Site)
5 Months ago
UXBERT Labs - Senior Data Engineer

UXBERT Labs

Riyadh, Riyadh Province, Saudi Arabia (Hybrid)
2 Months ago
The Walt Disney Company - Decision Science Undergraduate Intern

The Walt Disney Company

Lake Buena Vista, Florida, United States (On-Site)
21 Hours ago
PwC - IN-Manager_Big Data Engineer_Data and Analytics_Advisory_Bengaluru

PwC

Bengaluru, Karnataka, India (On-Site)
5 Months ago
N-iX - Senior Data Analyst/Power BI Engineer

N-iX

Ukraine (Remote)
1 Day ago
Dream Sports - SDE 2 - ML & Data Platform

Dream Sports

Mumbai, Maharashtra, India (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.


Hsinchu, Hsinchu City, Taiwan (On-Site)

Yokne'am Illit, North District, Israel (On-Site)

Seoul, South Korea (Hybrid)

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)

Ra'anana, Center District, Israel (On-Site)

Shanghai, Shanghai, China (On-Site)

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)

Be'er Sheva, South District, Israel (On-Site)

Santa Clara, California, United States (On-Site)

Santa Clara, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by NVIDIA

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug