Senior Staff Engineer - Kafka

8 Minutes ago • All levels • Software Development & Engineering

Job Summary

Job Description

This Senior Staff Engineer role at Nubank focuses on maintaining and operating a high-throughput Apache Kafka messaging platform, crucial for supporting over 105 million customers across Brazil, Mexico, and Colombia. The system processes trillions of messages monthly, underpinning mission-critical, low-latency operations. As Nubank expands, this role addresses significant scalability and operational challenges, including managing hundreds of Kafka clusters, thousands of brokers, and petabytes of daily data flow across multiple AWS accounts, ensuring high reliability and optimizing resource utilization.
Must have:
  • Operate and manage large-scale Apache Kafka infrastructure.
  • Ensure high reliability and minimal latency for petabytes of daily data.
  • Develop strategies for supporting and monitoring over a million Kafka topics.
  • Optimize resource utilization and implement cost-effective Kafka solutions across AWS accounts.
  • Drive evolution of asynchronous communication patterns beyond Kafka.
  • Implement solutions for performance, reliability, and issue containment with message growth.
  • Maintain and operate high-throughput messaging platforms on Apache Kafka.
  • Demonstrate proven expertise with Kafka in large-scale, mission-critical environments.
  • Possess experience with infrastructure spanning multiple AWS accounts.
  • Experience managing complex, large-scale data/messaging systems.
Perks:
  • Remote work, with quarterly trips to Sao Paulo to build relationships with coworkers
  • Top Tier Medical Insurance
  • Top Tier Dental and Vision Insurance
  • 20 days time off, 14 company holidays, and great culture that emphasizes work life balance
  • Life Insurance and AD&D
  • Extended maternity and paternity leaves
  • Nucleo - Our learning platform of courses
  • NuLanguage - Our language learning program
  • NuCare - Our mental health and wellness assistance program
  • 401K
  • Saving Plans - Health Saving Account and Flexible Spending Account

Job Details

About Nubank

Nu is the world’s largest digital banking platform outside of Asia, serving over 105 million customers across Brazil, Mexico, and Colombia. The company has been leading an industry transformation by leveraging data and proprietary technology to develop innovative products and services. Guided by its mission to fight complexity and empower people, Nu caters to customers’ complete financial journey, promoting financial access and advancement with responsible lending and transparency. The company is powered by an efficient and scalable business model that combines low cost to serve with growing returns. Nu’s impact has been recognized in multiple awards, including Time 100 Companies, Fast Company’s Most Innovative Companies, and Forbes World’s Best Banks. Learn more: https://international.nubank.com.br/careers/

About the Role

We are opening this position in the US to find an engineer with a highly specialized skill set that is critical to our current needs. The messaging team is responsible for maintaining and operating a high-throughput messaging platform built on Apache Kafka. This system processes trillions of messages every month and underpins mission-critical, low-latency systems used across clusters. This role is critical for the future success of the product.

As Nubank rapidly expands to support billions of customers across multiple countries, over 100 financial products, and 100+ business platforms, running on infrastructure spanning over multiple AWS accounts, we face significant scalability and operational challenges. Our current infrastructure includes approximately 200 Kafka clusters, 1500 brokers, more than 200k topics, and processes 300 TB of data daily. This role will directly address challenges such as:

  • Operating and managing over 200 Kafka clusters efficiently.
  • Supporting and monitoring over 1 million Kafka topics.
  • Handling over 1PB of data flowing through Kafka daily, ensuring minimal latency and high reliability.
  • Optimizing resource utilization and controlling costs in a smart, sustainable way.
  • Extending asynchronous communication patterns beyond Kafka to meet future needs.
  • Maintaining performance, reliability, and containing blast radius as message volume grows exponentially.

Our challenges:

  • In this role, you will be responsible for tackling the key challenges associated with our rapidly expanding, high-throughput Apache Kafka messaging platform. Your main activities and goals will include:
  • Operating and managing a large-scale Kafka infrastructure, including hundreds of clusters and thousands of brokers.
  • Ensuring high reliability and minimal latency for systems processing petabytes of data daily.
  • Developing strategies to efficiently support and monitor over a million Kafka topics.
  • Optimizing resource utilization and implementing cost-effective solutions for our Kafka ecosystem across multiple AWS accounts.
  • Driving the evolution of asynchronous communication patterns, potentially extending beyond Kafka to meet future needs.
  • Implementing solutions to maintain performance and reliability and contain potential issues as message volume grows exponentially.

We are looking for a person who has

  • Essential skills and experience in maintaining and operating high-throughput messaging platforms built on Apache Kafka.
  • Proven expertise with Kafka in large-scale, mission-critical environments.
  • Experience with infrastructure spanning multiple AWS accounts.
  • Experience in companies that manage similar complex, large-scale data/messaging systems.
  • A specialized skill set related to the challenges outlined above.

Benefits

  • Remote work, with quarterly trips to Sao Paulo to build relationships with coworkers.
  • Top Tier Medical Insurance
  • Top Tier Dental and Vision Insurance
  • 20 days time off, 14 company holidays, and great culture that emphasizes work life balance.
  • Life Insurance and AD&D
  • Extended maternity and paternity leaves
  • Nucleo - Our learning platform of courses
  • NuLanguage - Our language learning program
  • NuCare - Our mental health and wellness assistance program
  • Extended maternity and paternity leaves
  • 401K
  • Saving Plans - Health Saving Account and Flexible Spending Account

Role Location

  • Our Nu Way of Working Our work model is hybrid and has cycles that can be from two to three months according to the business of expertise. For every eight or twelve weeks of remote work, one will be at the office.

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in United States

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Software Development & Engineering Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Nubank was born in 2013 with the mission to fight against the complexity of the financial market to help our customers regain control of their financial lives. We have spent 11 years dedicated to bringing very simple ideas to places no one has ever taken them. For us, past success does not guarantee the future, which is why every day is “Day 1.” Being part of Nubank is embarking on a long-term journey where we know each challenge sparks creativity and innovation, where obstacles become opportunities to go a little further. Recently, we reached the milestone of 100 million customers globally, a significant achievement in our journey, but we know it wasn’t just the customers who chose us. We have over 8,000 Nubankers who choose to work with us daily.

State Of São Paulo, Brazil (Hybrid)

United States (Remote)

State Of São Paulo, Brazil (Hybrid)

State Of São Paulo, Brazil (Hybrid)

Mexico City, Mexico (Hybrid)

Bogota, Colombia (Hybrid)

Bogotá, Bogota, Colombia (On-Site)

Mexico City, Mexico (On-Site)

Mexico City, Mexico (Hybrid)

View All Jobs

Get notified when new jobs are added by nubank

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug