Software Engineer III, Infrastructure, Conduit Flume Pipelines

3 Weeks ago • 2-4 Years • Full Stack Development • Backend Development

Job Summary

Job Description

This Software Engineer III role focuses on Conduit Flume Pipelines (CFP) within Google's infrastructure. Responsibilities include building components for deploying, scheduling, and monitoring Flume jobs; delivering new features to expand CFP's use cases, especially for ML data processing; building core orchestration libraries in C++; defining and extending APIs in C++, Python, Java, and Go; working with infrastructure teams to integrate with their APIs; and contributing to Dreampipe. The ideal candidate will have experience with C++, object-oriented programming, and Flume pipelines, along with a strong understanding of data structures, algorithms, and distributed systems.
Must have:
  • C++ development experience
  • Object-oriented programming expertise
  • Flume pipeline experience
  • Data structures & algorithms knowledge
  • Distributed systems understanding
Good to have:
  • ML data processing workflow experience
  • Experience with Dreampipe

Job Details

Minimum qualifications:

  • Bachelor’s degree or equivalent practical experience.
  • 2 years of experience with software development in one or more programming languages, or 1 year of experience with an advanced degree.
  • 2 years of experience with data structures or algorithms.
  • 2 years of experience with developing infrastructure, distributed systems or networks, or with compute technologies, storage or hardware architecture.
  • Experience developing software applications using the C++ programming language.
  • Experience with object-oriented programming, templates, and the Standard Template Library (STL).

Preferred qualifications:

  • Experience in operating and managing flume pipelines.
  • Experience with ML data processing workflows.

About the job

Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.

The Core team builds the technical foundation behind Google’s flagship products. We are owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure at Google. These are the essential building blocks for excellent, safe, and coherent experiences for our users and drive the pace of innovation for every developer. We look across Google’s products to build central solutions, break down technical barriers and strengthen existing systems. As the Core team, we have a mandate and a unique opportunity to impact important technical decisions across the company.

Responsibilities

  • Build components of a managed system that handles deployment, scheduling, launching, monitoring and debugging of Flume jobs.
  • Deliver new features to expand the set of use cases that Conduit Flume Pipelines (CFP) can serve, including support for data processing for Machine Learning (ML).
  • Build core orchestration libraries in C++, define and extend public Application Programming Interface (APIs) in C++, Python, Java and Go.
  • Work with the underlying infrastructure teams (e.g., Conduit, Flume, Data Engineering Console, Table Service) to integrate with their APIs, define requirements for new features and make code contributions to their codebases.
  • Make smaller contributions to Dreampipe to continue supporting the most important new use cases.

Similar Jobs

Google - Senior Software Engineer, Google Messages Wear, Android

Google

Kraków, Lesser Poland Voivodeship, Poland (On-Site)
1 Week ago
Luxoft - Neoxam Consultant

Luxoft

Sydney, New South Wales, Australia (On-Site)
5 Months ago
Playtika - JavaScript Technical Lead

Playtika

Ukraine (On-Site)
4 Months ago
Playrix - Senior Engineering Manager

Playrix

Montenegro (Remote)
6 Months ago
London stock Exchange - Developer Platforms Engineer

London stock Exchange

Bucharest, Bucharest, Romania (On-Site)
20 Hours ago
Milestone - Senior Software Engineer

Milestone

Portland, Oregon, United States (Remote)
1 Month ago
Google - Software Engineer, Cloud AI, Early Career

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
3 Weeks ago
DOTSOFT SA - Systems Architect | Technical Lead for Digital Platforms

DOTSOFT SA

Greece (Remote)
2 Weeks ago
Maersk Careers - Senior Software Engineer

Maersk Careers

Shanghai, Shanghai, China (On-Site)
7 Months ago
Google - Senior Software Engineer, Core Data, Data Indexing

Google

Bengaluru, Karnataka, India (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Aristocrat Gaming - Fullstack Engineer

Aristocrat Gaming

London, England, United Kingdom (Hybrid)
4 Weeks ago
Bohemia Interactive - QA Automation Engineer - DayZ

Bohemia Interactive

Prague, Prague, Czechia (On-Site)
3 Months ago
ARHS - Senior System Integration & Performance Testing Engineer

ARHS

Athens, Greece (On-Site)
1 Month ago
Velotio Technologies - Lead Engineer (Java)

Velotio Technologies

Pune, Maharashtra, India (Remote)
1 Month ago
ByteDance - Senior Backend Software Engineer - Customer Service Platform

ByteDance

Seattle, Washington, United States (On-Site)
1 Month ago
McDonald's Corporation - Senior Manager, Mobile Software Engineering & Innovation

McDonald's Corporation

Mexico City, Mexico City, Mexico (On-Site)
2 Weeks ago
Workato - Technical Architect

Workato

Hyderabad, Telangana, India (On-Site)
1 Week ago
Google - Software Engineer III, Full Stack, Learning and Education

Google

Mexico City, Mexico City, Mexico (On-Site)
3 Weeks ago
ION - Technical Consultant - Endur

ION

Jersey City, New Jersey, United States (On-Site)
7 Months ago
Meta - Software Engineer, Android

Meta

Menlo Park, California, United States (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Zürich, Zurich, Switzerland

Powerintegration - Technician, Supervisor Prototype Manufacturing Lab

Powerintegration

Biel/Bienne, Canton Of Bern, Switzerland (On-Site)
3 Months ago
Tesla - Automotive Mechatronics Technician Apprenticeship

Tesla

Zürich, Zurich, Switzerland (On-Site)
2 Months ago
Tesla - Area Parts Supervisor

Tesla

Cham, Zug, Switzerland (On-Site)
2 Months ago
Google - Sales Specialist, Go-To-Market, Alps, Google Cloud

Google

Zürich, Zurich, Switzerland (On-Site)
3 Weeks ago
PwC - Senior Associate / (Senior) Manager – Deals – Separation and Integration

PwC

Zürich, Zurich, Switzerland (On-Site)
7 Months ago
Niantic - Senior Software Engineer, Security

Niantic

Zürich, Zurich, Switzerland (Hybrid)
1 Month ago
PwC - Senior Manager Actuarial Services

PwC

Zürich, Zurich, Switzerland (On-Site)
7 Months ago
Google - Software Engineer III, Infobot, Tools and Automated Workflows

Google

Zürich, Zurich, Switzerland (On-Site)
1 Week ago
luxsoft - PL1 Developer

luxsoft

Zürich, Zurich, Switzerland (On-Site)
2 Days ago
Google - Security Engineer, Hardware Security, Cloud CISO

Google

Zürich, Zurich, Switzerland (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Full Stack Development Jobs

Google - Senior Software Engineer, Metrics and Business Intelligence

Google

Mexico City, Mexico City, Mexico (On-Site)
1 Week ago
Netflix - Software Engineer (L5) - Distributed Refactoring Platform

Netflix

Warsaw, Masovian Voivodeship, Poland (Hybrid)
6 Months ago
Google - Senior Solutions Acceleration Architect, Application

Google

Singapore (On-Site)
3 Weeks ago
Newzoo - Full Stack Python Developer

Newzoo

Amsterdam, North Holland, Netherlands (Hybrid)
1 Month ago
NVIDIA - Senior Software Architect - Data Center Systems

NVIDIA

Santa Clara, California, United States (Remote)
3 Months ago
Next Level Business Services - Software Design Engineer

Next Level Business Services

Redmond, Washington, United States (On-Site)
6 Months ago
Nagarro - Senior Engineer

Nagarro

Bengaluru, Karnataka, India (On-Site)
6 Months ago
Appirits - Web Engineer

Appirits

Shibuya, Tokyo, Japan (Hybrid)
1 Month ago
Vigaet - Full Stack Developer Internship

Vigaet

(On-Site)
6 Months ago
Warner Bros Games - Senior Software Engineer - Java

Warner Bros Games

Bengaluru, Karnataka, India (Hybrid)
2 Months ago

Get notifed when new similar jobs are uploaded

About The Company

A problem isn't truly solved until it's solved for all. Googlers build products that help create opportunities for everyone, whether down the street or across the globe. Bring your insight, imagination and a healthy disregard for the impossible. Bring everything that makes you unique. Together, we can build for everyone.

Fremont, California, United States (On-Site)

Bengaluru, Karnataka, India (On-Site)

Reston, Virginia, United States (On-Site)

Sunnyvale, California, United States (On-Site)

New Taipei, New Taipei City, Taiwan (On-Site)

Dublin, County Dublin, Ireland (On-Site)

San Jose, California, United States (On-Site)

Mexico City, Mexico City, Mexico (On-Site)

View All Jobs

Get notified when new jobs are added by Google

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug