Engineering Manager, Spark

1 Day ago • All levels

About the job

SummaryBy Outscal

Lead a team of engineers building & maintaining Spark-based data processing platforms at Netflix. Experience building and operationalizing Spark or similar distributed systems is a must. Proven track record of leading teams and delivering results under pressure is essential.

Netflix is one of the world’s leading entertainment services with 278 million paid memberships in over 190 countries enjoying TV series, films and games across a wide variety of genres and languages. Members can play, pause and resume watching as much as they want, anytime, anywhere, and can change their plans at any time.

The Role

Big Data Analytics Platform (BDAP) group is responsible for providing orchestration and compute technologies to ETL and to access nearly 1 ExaByte of data in the Netflix Warehouse. It is also responsible for the underlying table format, Iceberg (an industry standard table format and an open source project that originated in this team), metadata, security and the life-cycle of the data (janitoring, compacting etc).

The Big Data Compute Spark (BDCS) team provides high-performance, scalable, distributed data processing at rest with the customized-for-Netflix fork of Apache Spark on Hadoop to meet Netflix’s large scale data analytics and machine learning needs. This team of 7 people (and growing) is central to batch data processing and analytics in the Data Platform at Netflix. It provides support for Spark to ETL data into the Exabytes-scale data warehouse and a variety of ways (for casual and power users) to access that data using Spark.

The team is working on integrating Spark accelerators to further drive performance, evaluating containerization platforms to run Spark, building better batch and real time observability into Spark workflows and no/low touch ways to use Spark. 

This is a dream team of highly passionate and intelligent engineers who work well together. The team works on solving challenging problems at scale that have a huge impact on the Data Platform. The team has PMC members and committers who shape and contribute to open-source projects. 

What you will do:

  • Partner to deliver the vision, strategy, and adoption of current and future technologies

  • Form trusting cross-functional partnerships to align many engineering teams and ensure our solutions meet their needs, selflessly prioritizing work beyond the scope of your own domain

  • Build, scale, and grow a team of outstanding engineers, challenge them to bring their best selves to work every day, and deliver industry-leading results in stability, performance, and efficiency

  • Balance smart risks, investment in foundational technical work, paying off tech debt, and incremental improvements to deliver timely results across multiple critical strategies

What we are looking for:

  • An experienced engineering leader who can drive a team of amazing and geographically distributed engineers to do their best work while building and maintaining strong partnerships with peers and stakeholders

  • Ability to dive deep as needed to facilitate technical strategy trade-offs and zoom out to understand the big picture to shape the product

  • Experience building and running extremely reliable platform systems supporting high-scale and critical workloads, including running a 24x7 on-call, 99.95+% availability, fast response and short mean-time-to-restore service

  • Bring rigor to decision-making and drive down ambiguity with the right mix of collecting data, seeking input, farming for dissent, and making pilot investments to build understanding or confirm direction

  • Proven track record helping a team deliver timely results on critical priorities against many demands for their time, and partnering with product and engineering management to influence the right areas for investment

  • Experience building and leading software engineering teams with a focus on inclusion and diversity, ensuring that we can attract and retain the best talent from anywhere and that all voices are heard to inform our direction

  • Experience in building and operationalizing Spark or similarly complex Distributed Systems is a must.

A few more things about us:
As a team, we come from many different backgrounds and countries, and our fields of education range from the humanities to engineering to computer science and we strive to give people the opportunity to wear different hats, should they choose to. We strongly believe this diversity and agility has helped us build an inclusive and empathetic environment, and we are looking forward to adding your perspective to the mix!

Our culture is unique, and we tend to live by our values, so it’s worth learning more about Netflix . 
 

*****************


Compensation and Benefits
Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top-of-market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $190,000 - $920,000.

Netflix provides comprehensive benefits including Health Plans, Mental Health support, a 401(k) Retirement Plan with employer match, Stock Option Program, Disability Programs, Health Savings and Flexible Spending Accounts, Family-forming benefits, and Life and Serious Injury Benefits. We also offer paid leave of absence programs.  Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off. Full-time salaried employees are immediately entitled to flexible time off.

Netflix is a unique culture and environment.  Learn more .

We are an equal-opportunity employer and celebrate diversity, recognizing that diversity of thought and background builds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.

Job is open for no less than 7 days and will be removed when the position is filled.

$190.0K - $920.0K/yr (Outscal est.)
$555.0K/yr avg.
United States

About The Company

Netflix is one of the world's leading entertainment services with over 247 million paid memberships in over 190 countries enjoying TV series, films and games across a wide variety of genres and languages. Members can play, pause and resume watching as much as they want, anytime, anywhere, and can change their plans at any time.

California, United States (On-Site)

California, United States (On-Site)

California, United States (Remote)

California, United States (On-Site)

Los Gatos, United States Of America • Los Angeles, California, United States Of America (On-Site)

Los Gatos, United States Of America • New York, New York, United States Of America (On-Site)

Usa Remote • Los Gatos, California, United States Of America (Remote)

View All Jobs

Similar Jobs

Arkadium - Engineering Manager, AdTech

Portugal United Kingdom Spain (Remote)

Discord - Senior Engineering Manager, Realtime Infrastructure

California, United States (Hybrid)

Discord - Engineering Manager - Safety Platform

San Francisco, Ca Or U. (Remote)

Discord - Engineering Manager, Audio & Video Infrastructure

San Francisco, Ca Or U. (Remote)

ByteDance - Data Center System Software Engineering Manager

California, United States (On-Site)

Playtech - Engineering Manager

Poligona, Boulevard "tsarigradsko Shose" 115, Fia, Bulgaria (On-Site)

Rocket Science - Software Engineering Manager - Games

New York, United States (Hybrid)

Rocket Science - Software Engineering Manager - Games

New York, United States (Hybrid)

Toptracer - Software Engineering Manager

Stockholm County, Sweden (Hybrid)

Sonar Source - Engineering Manager

Texas, United States (On-Site)

Similar Skill Jobs

Warner Bros. Games - Digital Production Placement Intern

England, United Kingdom (Hybrid)

Arkadium - Engineering Manager, AdTech

Portugal United Kingdom Spain (Remote)

Gaming Innovation Group  - Head of Partnerships

United Kingdom (Remote)

Bungie - Marathon UI/UX Lead

Worldwide (Hybrid)

PTW - Chief Financial Officer - London

England, United Kingdom (On-Site)

Jobs in United States

Software Engineering Jobs

Warner Bros. Games - Digital Production Placement Intern

England, United Kingdom (Hybrid)

Take-Two Interactive - Senior Director, Workday

New York, United States (On-Site)

Arkadium - Engineering Manager, AdTech

Portugal United Kingdom Spain (Remote)

Rebellion - Location Manager

England, United Kingdom (On-Site)

Discord - Senior Software Engineer, Activities Studio

San Francisco, Ca Or U. (Remote)

Discord - Director, Workplace IT

California, United States (Hybrid)

Discord - Staff Software Engineer, Ads Serving (Quests)

San Francisco, Ca Or U. (Remote)

Discord - Software Engineer - Activity Feed

San Francisco, Ca Or U. (Remote)

Discord - Senior Software Engineer, Revenue Growth

San Francisco, Ca Or U. (Remote)

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug