Software Engineer, Conduit Flume Pipelines
Job Summary
Job Description
As a Software Engineer on the Conduit Flume Pipelines team, you'll build components for a managed system handling deployment, scheduling, launching, monitoring, and debugging of Flume jobs. You'll develop new features to expand Conduit Flume Pipelines' capabilities, build core orchestration libraries in C++, and define/extend public APIs in C++, Python, Java, and Go. Collaboration with infra teams (Conduit, Flume, Data Engineering Console, DPPE, Table Service) is crucial for API integration, defining feature requirements, and contributing directly to their codebases. You'll also contribute to Dreampipe to support key new use cases. This role requires a Bachelor's degree or equivalent experience and at least 2 years of software development experience.
Must have:
- Bachelor's degree or equivalent experience
- 2+ years software development experience
- Proficiency in Ruby, Python, Java, or C++
- Flume pipeline experience
- Build and manage Flume jobs
- Develop core orchestration libraries
Good to have:
- Experience with Engineering Tools and Infrastructure
6 skills required
Job Details
Minimum qualifications:
- Bachelor’s degree or equivalent practical experience.
- 2 years of experience in software development in one or more general purpose programming languages.
- Experience programming in one or more of the following languages: Ruby, Python, Java, C++.
Preferred qualifications:
- Experience with Engineering Tools and Infrastructure.
- Experience in running and managing Flume pipelines.
About the job
The Core team builds the technical foundation behind Google’s flagship products. We are owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure at Google. These are the essential building blocks for excellent, safe, and coherent experiences for our users and drive the pace of innovation for every developer. We look across Google’s products to build central solutions, break down technical barriers and strengthen existing systems. As the Core team, we have a mandate and a unique opportunity to impact important technical decisions across the company.Responsibilities
- Build components of a managed system that works on deployment, scheduling, launching, monitoring, and debugging of Flume jobs.
- Deliver new features to expand the set of use cases that Conduit Flume Pipelines can serve.
- Build core orchestration libraries in C++, define and extend public APIs in C++, Python, Java and Go.
- Work with the underlying infra teams (Conduit, Flume, Data Engineering Console, DPPE, Table Service) to integrate with their APIs, define requirements for new features, and make direct code contributions to their codebases.
- Make smaller contributions to Dreampipe to continue supporting the most important new use cases.
Similar Jobs
Axinous
San Jose, California, United States (Hybrid)
• 4 Months ago
Garena
Singapore (On-Site)
• 7 Months ago
Get notifed when new similar jobs are uploaded
Similar Skill Jobs
Amazon Games
Irvine, California, United States (On-Site)
• 7 Months ago
Next Level Business Services
Sunnyvale, California, United States (On-Site)
• 7 Months ago
Rockstar Games
New York, New York, United States (On-Site)
• 6 Months ago
Glean
Bengaluru, Karnataka, India (On-Site)
• 6 Months ago
Visa
Bengaluru, Karnataka, India (On-Site)
• 7 Months ago
![Actian - Senior Principal Software Engineer [gn] GERMANY](/_next/image?url=https%3A%2F%2Foutscal-lms-data.s3.ap-south-1.amazonaws.com%2Fcompany-image%2Fproduction%2Factian%2Factian_logo_10_16_2024__09_38_39.jpg&w=256&q=50){kind=logo}
Actian
Hamburg, Hamburg, Germany (Remote)
• 7 Months ago
Get notifed when new similar jobs are uploaded
Jobs in Zürich, Zurich, Switzerland
Tesla
Landquart, Grisons, Switzerland (On-Site)
• 4 Months ago
Microsoft
Zürich, Zurich, Switzerland (On-Site)
• 4 Months ago
PwC
Zürich, Zurich, Switzerland (On-Site)
• 8 Months ago
Get notifed when new similar jobs are uploaded
Backend Development Jobs
Activision
San Francisco, California, United States (On-Site)
• 6 Months ago
Applike Group
Hamburg, Hamburg, Germany (Hybrid)
• 7 Months ago
Salesforce
Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
• 8 Months ago
Get notifed when new similar jobs are uploaded
