Principal Software Engineer - Ad Platform

2 Months ago • 10 Years + • Data Analyst • $184,300 PA - $270,700 PA

Job Summary

Job Description

The Principal Software Engineer will architect and implement scalable data solutions on the Databricks platform for Disney's Ad Platforms Data platform. This role involves designing ETL/ELT pipelines, optimizing data workflows, ensuring data security and governance, and collaborating with cross-functional teams. Responsibilities include leading the adoption of best practices in data warehousing, lakehouse architecture, and metadata management; overseeing Airflow integration; and driving implementation of Unity Catalog. The ideal candidate will have extensive experience with Databricks, Spark, Flink, Airflow, and Delta Lake, as well as expertise in AWS cloud services and data compliance.
Must have:
  • 10+ years big data engineering experience
  • Expertise in Databricks, Spark, Delta Lake
  • AWS cloud services knowledge
  • Proficiency in Python/Java/Scala
  • Data warehousing & lakehouse architecture
  • Data governance & compliance expertise
Good to have:
  • Media advertising domain experience
  • Experience with Iceberg format
  • Metadata management tools expertise
  • Team leadership experience

Job Details

Job Summary:

On any given day at Disney Entertainment & ESPN Technology, we’re reimagining ways to create magical viewing experiences for the world’s most beloved stories while also transforming Disney’s media business for the future. Whether that’s evolving our streaming and digital products in new and immersive ways, powering worldwide advertising and distribution to maximize flexibility and efficiency, or delivering Disney’s unmatched entertainment and sports content, every day is a moment to make a difference to partners and to hundreds of millions of people around the world.

A few reasons why we think you’d love working for Disney Entertainment & ESPN Technology

  • Building the future of Disney’s media business: DE&E Technologists are designing and building the infrastructure that will power Disney’s media, advertising, and distribution businesses for years to come.

  • Reach & Scale: The products and platforms this group builds and operates delight millions of consumers every minute of every day – from Disney+ and Hulu, to ABC News and Entertainment, to ESPN and ESPN+, and much more.

  • Innovation: We develop and execute groundbreaking products and techniques that shape industry norms and enhance how audiences experience sports, entertainment & news.

The products and platforms the Ad Platforms group builds and operates delight millions of consumers every minute of every day – from Disney+ and Hulu, to ABC News and Entertainment, to ESPN and ESPN+, and much more.  We develop and execute groundbreaking products and techniques that shape industry norms and enhance how audiences experience sports, entertainment & news.

The Ad Platforms team is responsible for building and operating the technology that supports Disney’s industry-leading, multi-billion dollar global advertising business. Working across all of Disney’s streaming brands and linear channels, this team’s scope is wide ranging - they impact everything from core infrastructure development that powers Disney’s proprietary ad server, to reimagining viewer-first ad experiences and formats. This team’s award-winning, in-house built technologies are the backbone of Disney’s ad business and a critical element of the Company’s streaming future.

The Disney Ad Platforms Data team is at the forefront of building a modern advertising data platform to support our organization’s data, analytics and business intelligence needs. Our mission is to enable data engineers, data scientists, analysts, and decision-makers to derive value from our data ecosystem.

Job Summary:

We are seeking a Principal Software Engineer with extensive experience in the Databricks platform and advanced big data engineering practices. This role involves architecting and implementing scalable, secure, and high-performing data solutions to power the Ad Platforms Data platform and support mission-critical data initiatives, including real-time and batch processing pipelines.

As a hands-on technical leader, you will design data architectures that scale to handle terabytes-per-hour throughput, ensuring optimal performance, compliance, and alignment with business needs. You will play a pivotal role in guiding teams, collaborating across departments, and driving innovation. 

Responsibilities:

Technical Leadership and Architecture

  • Architect and implement highly scalable, secure, and efficient ETL/ELT pipelines for both real-time and batch data processing.

  • Design complex, multi-source data ecosystems and ensure seamless integration with tools like Databricks, Delta Lake, and Snowflake.

  • Optimize data pipelines and workflows for performance, cost, and reliability, leveraging cutting-edge tools like Airflow, Spark, Flink, and Databricks Delta Sharing.

  • Lead the adoption of best practices in data warehousing, lakehouse architecture, and metadata management.

  • Champion data security, privacy, and governance, implementing frameworks for lineage tracking, cataloging, and compliance (e.g., SOx, GDPR, CCPA).

  • Drive implementation of Unity Catalog for data governance and cross-account querying.

Platform Development and Operational Excellence

  • Able to assist in the set up and manage AWS environments, including S3, IAM Roles, Network Peering, and other infrastructure components.

  • Oversee Airflow integration with Databricks Spark clusters, manage user access, and implement robust secret management solutions.

  • Lead the CI/CD process with gated code releases, ensuring code quality and scalable deployment processes.

  • Implement innovative solutions, from the exciting to the mundane; such as Databricks UniForm (Iceberg, Delta) format integration with Snowflake and basic tools to support cost projection, etc.

Data Engineering and Optimization

  • Optimize Spark Streaming and ETL workflows, leveraging deep expertise in Spark performance tuning.

  • Ensure best practices in data ingestion, transformation, and loading, focusing on highly performant and governed data models.

  • Drive improvements in Delta Lake ecosystems and lakehouse architecture, including Delta Sharing for distributed querying.

Collaboration and Communication

  • Collaborate closely with cross-functional teams, including Ad Server architects, to align on data strategies and designs.

  • Foster open communication during architectural discussions, confidently defending decisions while welcoming feedback.

  • Bridge the gap between diverse team requirements and technical capabilities to deliver cohesive, scalable solutions.

Innovation and Mentorship

  • Stay ahead of emerging trends in big data technologies, cloud platforms (AWS, Azure, GCP), and distributed computing.

  • Mentor engineers and architects, fostering a culture of innovation, knowledge sharing, and technical excellence.

Basic Qualifications

  • Bachelor’s or Master's degree in Computer Science, Software Engineering or related technical discipline is highly desirable

  • 10+ years of experience in software engineering with a focus on big data platforms.

  • Proven expertise in building scalable systems capable of processing terabytes of data per hour.

  • Hands-on experience with Databricks, Spark, Flink, Airflow, and Delta Lake ecosystems.

  • Proficiency in Python (including PySpark), Java, or Scala.

  • Extensive knowledge of data warehousing, data lakes, and real-time streaming systems.

  • Hands-on experience with AWS cloud services and data solutions (e.g., S3, Redshift, Athena, IAM).

  • Deep understanding of data compliance (e.g., SOX, GDPR) and governance.

  • Strong skills in data lineage, cataloging, and metadata management.

Preferred Qualifications

  • Experience in the media advertising domain, including linear and digital Ad Sales.

  • Expertise with tools like ERStudio, ERWin, Alation, or Collibra for metadata management.

  • Familiarity with Iceberg format integration and other modern data formats.

  • Certifications in Databricks, AWS, or relevant big data technologies.

  • Ability to manage/lead a team of high-performing data engineers and architects.

  • Strong curiosity about how Disney delivers the Magic and a desire to be a part of it

#disneytech


The hiring range for this position in Los Angeles, CA is between $184,300 – $247,100 year and San Franscico, CA is between $201,900 - $270,700 and Seattle, WA is between $193,100 - $258,900. The base pay actually offered will take into account internal equity and also may vary depending on the candidate’s geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered.

Similar Jobs

Meta - Software Engineer, Infrastructure

Meta

Burlingame, California, United States (Remote)
5 Months ago
Ello - Product Engineer (ML & Mobile)

Ello

San Francisco, California, United States (On-Site)
1 Month ago
Google - Student Researcher, BS/MS, Winter/Summer 2025

Google

Ann Arbor, Michigan, United States (On-Site)
5 Months ago
Canva - Senior Backend Engineer (Java) - Content Growth, Global Content & Discovery

Canva

Sydney, New South Wales, Australia (Remote)
1 Week ago
ByteDance - Research Scientist in Multimodal Graduate (Applied Machine Learning) - 2024 Start (PhD)

ByteDance

San Jose, California, United States (On-Site)
5 Months ago
N-iX - Strong Middle Data Engineer (Azure Databricks)

N-iX

Poland (Remote)
4 Weeks ago
ByteDance - Data Scientist

ByteDance

San Jose, California, United States (On-Site)
2 Months ago
Google - Data Transformation Lead, Specialty Retail

Google

San Francisco, California, United States (On-Site)
4 Days ago
InMobiInMobi - Lead – Business Analytics & Insights (Internal Clients)

InMobiInMobi

Bengaluru, Karnataka, India (On-Site)
5 Months ago
Microsoft - Senior Security Data Scientist

Microsoft

Redmond, Washington, United States (On-Site)
5 Days ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Technorizen Software Solutions - Cross Platform Mobile App. Developer - (Ionic, React native, Xamarin)

Technorizen Software Solutions

Indore, Madhya Pradesh, India (On-Site)
9 Months ago
Microsoft - Senior Incident Response Engineer

Microsoft

Tel Aviv-Yafo, Tel Aviv District, Israel (Remote)
5 Days ago
Google - Senior Software Engineer, Full Stack

Google

(On-Site)
5 Months ago
Razer - Senior API Developer

Razer

Singapore (On-Site)
6 Months ago
Samsung Semiconductor - Staff Software Engineer – Platform

Samsung Semiconductor

San Jose, California, United States (Hybrid)
1 Month ago
CloudHire - Database Solution Architect

CloudHire

Gurugram, Haryana, India (Remote)
3 Weeks ago
Google - Product Solutions Engineer, Billing, Tech CoE

Google

Sunnyvale, California, United States (On-Site)
4 Days ago
Simple Viral Games - Lead Software Developer

Simple Viral Games

Bengaluru, Karnataka, India (On-Site)
8 Months ago
ByteDance - Backend Engineer Intern, Video-On-Demand

ByteDance

Singapore (On-Site)
3 Weeks ago
ByteDance - Data Analyst - Global Payment

ByteDance

Singapore (On-Site)
3 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Glendale, California, United States

Microsoft - Research Intern - AI for Domains

Microsoft

Redmond, Washington, United States (On-Site)
2 Days ago
Spellbrush - AI Infrastructure Engineer

Spellbrush

San Francisco, California, United States (On-Site)
1 Month ago
Floor 84 Studio - Summer Internship - Video Game Design & Development

Floor 84 Studio

Los Angeles, California, United States (On-Site)
4 Months ago
Keywords Studios - Senior VIP Account Manager

Keywords Studios

United States (Remote)
2 Weeks ago
ByteDance - Backend Software Engineer Intern (PDI-CSP-FE-i18n) - 2025 Summer (BS/MS)

ByteDance

Seattle, Washington, United States (On-Site)
3 Months ago
Trek - Part Time Service Technician and Bike Builder

Trek

Austin, Texas, United States (On-Site)
2 Months ago
Axon - Senior Security Engineer

Axon

Scottsdale, Arizona, United States (Hybrid)
4 Months ago
ByteDance - Central Strategy and Sales Policy Associate - Monetization Strategy & Operations

ByteDance

New York, New York, United States (On-Site)
3 Weeks ago
Google - Staff Software Engineer, YouTube

Google

Mountain View, California, United States (On-Site)
4 Days ago
Google - OTT Live Video Engineer

Google

New York, New York, United States (On-Site)
3 Days ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

N-iX - Senior Data Engineer

N-iX

Medellín, Antioquia, Colombia (Flexible)
1 Week ago
Playrix - Senior Data Analyst (Game)

Playrix

Montenegro (Remote)
6 Months ago
ION - Data Engineer/Analyst - Graduate Development Program

ION

Pisa, Tuscany, Italy (On-Site)
6 Months ago
Next Level Business Services - Google Analytics

Next Level Business Services

San Antonio, Texas, United States (On-Site)
6 Months ago
NVIDIA - Senior C++ Software Engineer - Apache Spark Solution

NVIDIA

Beijing, Beijing, China (On-Site)
1 Month ago
GoTo Group - FP&A - Data Analyst

GoTo Group

Jakarta, Jakarta, Indonesia (On-Site)
5 Months ago
Tesla - Data Engineer Internship

Tesla

North Holland, Netherlands (On-Site)
2 Months ago
Epic Games - Analytics Lead (Gameplay)

Epic Games

Toronto, Ontario, Canada (On-Site)
3 Months ago
DraftKings - Senior Data Engineer, Data Enablement

DraftKings

Boston, Massachusetts, United States (On-Site)
2 Weeks ago
Tonies - Working Student Product Data Management

Tonies

Düsseldorf, North Rhine-Westphalia, Germany (Remote)
3 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

From classic animated features and exhilarating theme park attractions to cutting edge sports coverage, and the hottest shows on television, The Walt Disney Company has been making magic since 1923, creating unforgettable stories that connect with audiences around the world. And we’re just getting started!

The key to our success…. The Cast, Crew, Imagineers and Employees who honor Disney’s rich legacy by stretching the bounds of imagination to create the never-before-seen, bringing unparalleled entertainment experiences to people of all ages. Begin a career that delivers unparalleled creative content and experiences to audiences around the world and just imagine the stories you could be part of…

What is #LifeAtDisney like? It’s a series of magical moments with cast members and employees developing and telling our stories in the most innovative ways. Whether it’s a day spent as a Disney VoluntEAR, or celebrating the release of a new interactive experience, retail product or movie, our days are filled with the knowledge that we are creating entertainment experiences the whole family can enjoy. Follow @DisneyCareers on Facebook, Twitter and Instagram for a peek behind-the-curtain, and discover how you could connect to a world of stories with Disney!

Montévrain, Île-de-France, France (On-Site)

Glendale, California, United States (Hybrid)

Île-de-France, France (On-Site)

São Paulo, State Of São Paulo, Brazil (On-Site)

Lake Buena Vista, Florida, United States (On-Site)

Glendale, California, United States (On-Site)

Burbank, California, United States (Remote)

Glendale, California, United States (Remote)

Anaheim, California, United States (On-Site)

Montévrain, Île-de-France, France (On-Site)

View All Jobs

Get notified when new jobs are added by The Walt Disney Company

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug