Engineering Manager - Ads Experimentation Platform

Reddit Ads Marketplace is a dynamic and complicated environment where millions of engagements among Reddit users, advertisers and the platform occur every second. It’s critical for us to extract useful and statistically significant signals from the massive volume of events to guide the product developments and decisions in a scientific way. The Ads Experimentation Platform (AEXP) team builds tools for our engineers and scientists to accurately test new products and efficiently verify new ideas in the ads marketplace. Have you heard about Budget Cannibalization? Have you wondered why a user-split A/B testing is insufficient in testing products in the Ads marketplace? How can we make the most out of rare and sparse samples (e.g. conversion events)? How can I explore 100 algorithmic candidates at the same time? These are the typical problems we are dealing with on a daily basis. Meanwhile, Ads Pacing plays a crucial role in both supporting the dynamism of the marketplace and accommodating the budget-aware A/B experiments, therefore our team is also committed to building scalable and flexible pacing infrastructures that improves the market efficiency and budget-aware testing capacity.

 

As the manager of the Ads Experimentation Platform team, you will work closely with engineers, data science partners and product leaders to improve the capacity and efficiency of existing experimentation tools, research and build new testing methodologies and tools, and lead the experimentation practices across the Ads marketplace teams. You will be responsible for the quality and technical approach within the team; partner with other managers in direction setting, planning, and overseeing engineering designs and executions; establish and contribute to the group’s culture and processes. You will collaborate with cross-functional engineering, product, and data science teams to ensure that our customers’ voices are represented in your work..

Some of the project highlights include:

• An A/B testing framework that split both users and ad budgets to create parallel and mutually exclusive control/treatment sub ads marketplaces,
• An exploration tool that can run multiple (hundreds of) A/B testing pairs of campaigns over multiple algorithmic candidates for pacing,
• Partitioning the pacing and budgeting systems to accommodate the growing number of advertisers and maintain the refreshing rate above once per minute.

Responsibilities

• Coach, Motivate, Build, Hire, and Lead a world-class team of passionate engineers with a knack for high scale/high performance Ads delivery systems
• Lead, coordinate, and execute a coherent vision for a ruthlessly prioritized infrastructure roadmap based on business needs
• Set and support a culture of data-informed decision making, with efficient processes and strong transparency
• Facilitate the collaboration between the different product/vertical teams making use of Ads Infrastructure and the partner and core platform teams across Reddit
• Participating in design and coding reviews: You will review work by your team members and provide feedback to ensure that it meets the team's standards for quality, performance and best practices
• Collaborating with cross-functional team leads (EMs, PMs, DSs)  to understand business requirements and translate them into technical directions for the Ads Infrastructure roadmap in conjunction with the rest of the Ads Foundations teams

Required Qualifications:

• 8+ years of industry experience as a SWE
• 3+ years managing (including coaching, mentoring, developing) engineering teams
• 2+ years in the capacity of tech lead in charge of systems & architecture design
• Experience with at least two general programming languages such as Python, Go, Scala, Java, C++
• Preferred experience in one or more of the following areas: Message Queuing Services (Kafka, Amazon Simple Queue Service), Data Processing Frameworks (Apache Spark, Apache Flink), Key Value Stores (Redis, DynamoDB, Memcached, Riak), Document based DBs (MongoDB, Cassandra), Container Orchestration (Kubernetes, Mesos), ML Ops ((ML | Tensor | Kube)flow)
• Excellent communication skills, both written and verbal, and the ability to work effectively with product managers, data scientists, and other stakeholders

Preferred Qualifications:

• Experience managing Infrastructure or Platform-y teams
• Experience with Ads
• Experience with Technical Program Management
• Industry relevant or scientific contribution in the field of DevOps, Infrastructure, High scalability/Big Data syste