Software Engineer III, Infrastructure, Conduit Flume Pipelines

1 Month ago • 2-4 Years • Full Stack Development • Backend Development

Job Summary

Job Description

This Software Engineer III role focuses on Conduit Flume Pipelines (CFP) within Google's infrastructure. Responsibilities include building components for deploying, scheduling, and monitoring Flume jobs; delivering new features to expand CFP's use cases, especially for ML data processing; building core orchestration libraries in C++; defining and extending APIs in C++, Python, Java, and Go; working with infrastructure teams to integrate with their APIs; and contributing to Dreampipe. The ideal candidate will have experience with C++, object-oriented programming, and Flume pipelines, along with a strong understanding of data structures, algorithms, and distributed systems.
Must have:
  • C++ development experience
  • Object-oriented programming expertise
  • Flume pipeline experience
  • Data structures & algorithms knowledge
  • Distributed systems understanding
Good to have:
  • ML data processing workflow experience
  • Experience with Dreampipe

Job Details

Minimum qualifications:

  • Bachelor’s degree or equivalent practical experience.
  • 2 years of experience with software development in one or more programming languages, or 1 year of experience with an advanced degree.
  • 2 years of experience with data structures or algorithms.
  • 2 years of experience with developing infrastructure, distributed systems or networks, or with compute technologies, storage or hardware architecture.
  • Experience developing software applications using the C++ programming language.
  • Experience with object-oriented programming, templates, and the Standard Template Library (STL).

Preferred qualifications:

  • Experience in operating and managing flume pipelines.
  • Experience with ML data processing workflows.

About the job

Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.

The Core team builds the technical foundation behind Google’s flagship products. We are owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure at Google. These are the essential building blocks for excellent, safe, and coherent experiences for our users and drive the pace of innovation for every developer. We look across Google’s products to build central solutions, break down technical barriers and strengthen existing systems. As the Core team, we have a mandate and a unique opportunity to impact important technical decisions across the company.

Responsibilities

  • Build components of a managed system that handles deployment, scheduling, launching, monitoring and debugging of Flume jobs.
  • Deliver new features to expand the set of use cases that Conduit Flume Pipelines (CFP) can serve, including support for data processing for Machine Learning (ML).
  • Build core orchestration libraries in C++, define and extend public Application Programming Interface (APIs) in C++, Python, Java and Go.
  • Work with the underlying infrastructure teams (e.g., Conduit, Flume, Data Engineering Console, Table Service) to integrate with their APIs, define requirements for new features and make code contributions to their codebases.
  • Make smaller contributions to Dreampipe to continue supporting the most important new use cases.

Similar Jobs

Google - Software Engineering Manager II, NodeOS

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
1 Month ago
Google - Software Engineer III, Full Stack, Google Cloud

Google

(On-Site)
6 Months ago
Microsoft - Software Engineer II

Microsoft

Redmond, Washington, United States (On-Site)
1 Month ago
Google - Business Analyst, Creator Partnerships

Google

New York, New York, United States (On-Site)
1 Month ago
Next Level Business Services - Java/J2EE Developer

Next Level Business Services

Tampa, Florida, United States (On-Site)
7 Months ago
Microsoft - Engineering Manager

Microsoft

Redmond, Washington, United States (On-Site)
1 Month ago
Innoactive - Software Engineer (.NET)

Innoactive

(Remote)
2 Months ago
Next Level Business Services - Web Development

Next Level Business Services

Moline, Illinois, United States (On-Site)
7 Months ago
Nagarro - Principal Engineer

Nagarro

Singapore (On-Site)
7 Months ago
Google - Software Engineering Manager II

Google

Sunnyvale, California, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Capgemini - Java

Capgemini

Hyderabad, Telangana, India (On-Site)
2 Weeks ago
Schbang - Senior HubSpot Developer

Schbang

Mumbai, Maharashtra, India (On-Site)
3 Months ago
LeoVegas - Backend Engineer - Regulatory Reporting

LeoVegas

Warsaw, Masovian Voivodeship, Poland (Hybrid)
2 Months ago
The Walt Disney Company - Senior Software Engineer - Activation Team

The Walt Disney Company

Santa Monica, California, United States (On-Site)
1 Month ago
SmileGate - Platform Division Member Authentication Backend Developer

SmileGate

Seongnam-si, Gyeonggi-do, South Korea (On-Site)
4 Months ago
Playrix - Director of Engineering

Playrix

Armenia (Remote)
7 Months ago
Cognite - Senior Solutions Architect

Cognite

Austin, Texas, United States (Hybrid)
2 Weeks ago
ION - Senior IT Architect, Italy

ION

Italy (Hybrid)
7 Months ago
QuinStreet - Java Developer

QuinStreet

Pune, Maharashtra, India (Hybrid)
2 Weeks ago
GoTo Group - Lead Software Engineer - Identity Platform

GoTo Group

Bengaluru, Karnataka, India (Hybrid)
5 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Zürich, Zurich, Switzerland

Microsoft - Member of Technical Staff - AI Multimodal

Microsoft

Zürich, Zurich, Switzerland (On-Site)
1 Month ago
Tesla - HR Partner

Tesla

Zug, Zug, Switzerland (On-Site)
3 Months ago
Tesla - Parts Service Associate

Tesla

Studen, Canton Of Bern, Switzerland (On-Site)
3 Months ago
The Walt Disney Company - Disney Research Intern

The Walt Disney Company

Zürich, Zurich, Switzerland (On-Site)
6 Months ago
Tesla - Automotive Mechatronics Technician

Tesla

Cham, Zug, Switzerland (On-Site)
3 Months ago
PwC - Senior Associate - SAP Global Trade Services

PwC

Zürich, Zurich, Switzerland (On-Site)
7 Months ago
Microsoft - Member of Technical Staff, AI Pre-Training

Microsoft

Zürich, Zurich, Switzerland (On-Site)
1 Month ago
Tesla - Automotive Service Technician

Tesla

Cadenazzo, Ticino, Switzerland (On-Site)
3 Months ago
PwC - Director in Life Sciences Quality Management

PwC

Zürich, Zurich, Switzerland (On-Site)
8 Months ago
PwC - Manager/ Senior Manager Financial Services - Technology Strategy & Transformation

PwC

Zürich, Zurich, Switzerland (On-Site)
8 Months ago

Get notifed when new similar jobs are uploaded

Full Stack Development Jobs

N-iX - Senior Python Engineer (With Scraping)

N-iX

Ukraine (Remote)
1 Month ago
Google - Senior Software Engineer, Infrastructure, Google Cloud AI

Google

Sunnyvale, California, United States (On-Site)
1 Month ago
Britive - SOFTWARE ENGINEER

Britive

San Francisco, California, United States (Remote)
6 Months ago
NVIDIA - Senior Full-Stack Web Applications Software Engineer

NVIDIA

Shanghai, Shanghai, China (On-Site)
1 Month ago
Google - Senior Bluetooth Firmware Engineer

Google

New Taipei, New Taipei City, Taiwan (On-Site)
1 Month ago
Rockstar Games - Full Stack Developer

Rockstar Games

New York, New York, United States (On-Site)
1 Month ago
Google - Software Engineer, Black Community Inclusion

Google

(On-Site)
6 Months ago
Google - Bluetooth Firmware Engineer

Google

New Taipei, New Taipei City, Taiwan (On-Site)
1 Month ago
Google - Senior Staff Software Engineer, Google Cloud Compute

Google

Sunnyvale, California, United States (On-Site)
1 Month ago
N-iX - Senior Frontend Engineer

N-iX

Ukraine (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded