Software Engineer III, Infrastructure, Conduit Flume Pipelines

8 Hours ago • 2-4 Years • Full Stack Development • Backend Development

Job Summary

Job Description

This Software Engineer III role focuses on Conduit Flume Pipelines (CFP) within Google's infrastructure. Responsibilities include building components for deploying, scheduling, and monitoring Flume jobs; delivering new features to expand CFP's use cases, especially for ML data processing; building core orchestration libraries in C++; defining and extending APIs in C++, Python, Java, and Go; working with infrastructure teams to integrate with their APIs; and contributing to Dreampipe. The ideal candidate will have experience with C++, object-oriented programming, and Flume pipelines, along with a strong understanding of data structures, algorithms, and distributed systems.
Must have:
  • C++ development experience
  • Object-oriented programming expertise
  • Flume pipeline experience
  • Data structures & algorithms knowledge
  • Distributed systems understanding
Good to have:
  • ML data processing workflow experience
  • Experience with Dreampipe

Job Details

Minimum qualifications:

  • Bachelor’s degree or equivalent practical experience.
  • 2 years of experience with software development in one or more programming languages, or 1 year of experience with an advanced degree.
  • 2 years of experience with data structures or algorithms.
  • 2 years of experience with developing infrastructure, distributed systems or networks, or with compute technologies, storage or hardware architecture.
  • Experience developing software applications using the C++ programming language.
  • Experience with object-oriented programming, templates, and the Standard Template Library (STL).

Preferred qualifications:

  • Experience in operating and managing flume pipelines.
  • Experience with ML data processing workflows.

About the job

Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.

The Core team builds the technical foundation behind Google’s flagship products. We are owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure at Google. These are the essential building blocks for excellent, safe, and coherent experiences for our users and drive the pace of innovation for every developer. We look across Google’s products to build central solutions, break down technical barriers and strengthen existing systems. As the Core team, we have a mandate and a unique opportunity to impact important technical decisions across the company.

Responsibilities

  • Build components of a managed system that handles deployment, scheduling, launching, monitoring and debugging of Flume jobs.
  • Deliver new features to expand the set of use cases that Conduit Flume Pipelines (CFP) can serve, including support for data processing for Machine Learning (ML).
  • Build core orchestration libraries in C++, define and extend public Application Programming Interface (APIs) in C++, Python, Java and Go.
  • Work with the underlying infrastructure teams (e.g., Conduit, Flume, Data Engineering Console, Table Service) to integrate with their APIs, define requirements for new features and make code contributions to their codebases.
  • Make smaller contributions to Dreampipe to continue supporting the most important new use cases.

Similar Jobs

ByteDance - Software Engineer Intern (Cloud Native Infrastructure)

ByteDance

San Jose, California, United States (On-Site)
3 Weeks ago
Google - Software Engineer, Early Career, Campus

Google

Bucharest, Bucharest, Romania (On-Site)
3 Months ago
ByteDance - Site Reliability Engineer, Compute Platform

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
Netflix - Machine Learning Software Engineer L4/L5

Netflix

United States (Remote)
3 Months ago
Google - Senior Software Engineer, Google Messages Wear, Android

Google

Kraków, Lesser Poland Voivodeship, Poland (On-Site)
9 Hours ago
Super - Senior Full-Stack Software Engineer ( Remote! )

Super

Toronto, Ontario, Canada (Remote)
5 Months ago
Onward Search - Java Developer

Onward Search

San Jose, California, United States (Hybrid)
3 Weeks ago
IO Interactive - Senior Frontend Developer

IO Interactive

İstanbul, İstanbul, Türkiye (Hybrid)
3 Weeks ago
Google - Uber Technical Lead, Spaces Engineering

Google

Hyderabad, Telangana, India (On-Site)
10 Hours ago
The Walt Disney Company - Senior Software Engineer - Rust

The Walt Disney Company

Seattle, Washington, United States (On-Site)
2 Days ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Push Gaming - Game Mathematician

Push Gaming

(Hybrid)
3 Weeks ago
Nolimit City - Backend developer

Nolimit City

Stockholm, Stockholm County, Sweden (On-Site)
5 Months ago
ByteDance - Software Engineer Intern (Recommendation Infrastructure)

ByteDance

San Jose, California, United States (On-Site)
3 Weeks ago
Microsoft - Software Engineer II

Microsoft

Hyderabad, Telangana, India (On-Site)
19 Hours ago
Canva - Backend Engineer (Java)

Canva

Sydney, New South Wales, Australia (Remote)
3 Weeks ago
Netomi - Software Engineer In Test III

Netomi

Gurugram, Haryana, India (Hybrid)
6 Months ago
ION - Java Developer, Budapest

ION

Budapest, Hungary (Hybrid)
6 Months ago
N-iX - Senior Fullstack Engineer (TypeScript)

N-iX

Ukraine (Hybrid)
2 Days ago
Saviynt - Lead Product Manager - Integration Framework

Saviynt

Bengaluru, Karnataka, India (Hybrid)
6 Months ago
Google - Engineering Manager, GKE AI, Experience and Ecosystem

Google

Seattle, Washington, United States (On-Site)
9 Hours ago

Get notifed when new similar jobs are uploaded

Jobs in Zürich, Zurich, Switzerland

GIANTS Software - Tools Programmer

GIANTS Software

Schlieren, Zurich, Switzerland (On-Site)
3 Months ago
Google - Senior Software Engineer, Full Stack, Protected Data Infrastructure

Google

Zürich, Zurich, Switzerland (On-Site)
9 Hours ago
The Walt Disney Company - Disney Research Intern

The Walt Disney Company

Zürich, Zurich, Switzerland (On-Site)
5 Months ago
Google - Software Engineer II, Account Threats

Google

Zürich, Zurich, Switzerland (On-Site)
11 Hours ago
PwC - Manager/Senior Manager - Operations and Supply Chain Management Consulting 80-100%

PwC

Zürich, Zurich, Switzerland (On-Site)
6 Months ago
Tesla - Automotive Mechatronician

Tesla

Zürich, Zurich, Switzerland (On-Site)
2 Months ago
PwC - Senior Associate - SAP Global Trade Services

PwC

Zürich, Zurich, Switzerland (On-Site)
6 Months ago
Tesla - Automotive Mechatronics/Automotive Specialist

Tesla

Studen, Canton Of Bern, Switzerland (On-Site)
2 Months ago
PwC - Auditor - Treasury and Commodity Trading

PwC

Geneva, Geneva, Switzerland (On-Site)
6 Months ago
Niantic - Senior Software Engineer, Security

Niantic

Zürich, Zurich, Switzerland (Hybrid)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Full Stack Development Jobs

Super - Senior Full-Stack Software Engineer ( Remote! )

Super

Portland, Oregon, United States (Remote)
5 Months ago
USE Insider - Front-end Developer (Javascript)

USE Insider

İstanbul, İstanbul, Türkiye (On-Site)
5 Months ago
Easygo - Software Development Engineer (Frontend)

Easygo

Melbourne, Victoria, Australia (On-Site)
3 Weeks ago
Warner Bros Games - Senior Software Engineer - Roku

Warner Bros Games

Bengaluru, Karnataka, India (Hybrid)
1 Month ago
Google - Software Engineer II, Shopping Ads

Google

Zürich, Zurich, Switzerland (On-Site)
8 Hours ago
Google - Software Engineer III, Front End, Google Cloud

Google

Ramat Gan, Tel Aviv District, Israel (On-Site)
10 Hours ago
Fairmatic - Senior Full Stack Engineer

Fairmatic

Tel Aviv-Yafo, Tel Aviv District, Israel (Hybrid)
6 Months ago
Super - Senior Full-Stack Software Engineer ( Remote! )

Super

Vancouver, British Columbia, Canada (Remote)
5 Months ago
Google - Software Engineer, People with Disabilities

Google

São Paulo, State Of São Paulo, Brazil (On-Site)
5 Months ago
Next Level Business Services - Java/C++ Developer

Next Level Business Services

Sunnyvale, California, United States (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

About The Company

A problem isn't truly solved until it's solved for all. Googlers build products that help create opportunities for everyone, whether down the street or across the globe. Bring your insight, imagination and a healthy disregard for the impossible. Bring everything that makes you unique. Together, we can build for everyone.

Bucharest, Bucharest, Romania (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Dublin, County Dublin, Ireland (On-Site)

Bengaluru, Karnataka, India (On-Site)

Sunnyvale, California, United States (On-Site)

Sunnyvale, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by Google

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug