Data Engineer - Video Architecture, LiveNet (Live Streaming Networks)

1 Week ago • All levels • Data Analyst

Job Summary

Job Description

ByteDance's Video and Edge department seeks a Data Engineer for its LiveNet (Live Streaming Networks) team. Responsibilities include designing and developing data processing architectures and ETL pipelines, architecting and maintaining efficient data pipelines, improving internal processes, and collaborating with cross-functional teams (CDN, Video Understanding, etc.). The role involves working with big data infrastructure and platforms to provide high-quality data to various teams, supporting core business metrics analysis. The ideal candidate possesses strong programming skills (Spark SQL, Python, NoSQL), understands data modeling and warehouse architecture, and has experience with big data processing software (Hadoop, Spark).
Must have:
  • Design and develop data processing architecture and systems
  • Build efficient and reliable data pipelines
  • Improve data quality and SLA management
  • Collaborate with cross-functional teams
  • Experience with Spark SQL, Python, or NoSQL databases
  • Understanding of data model design and data warehouse architecture
Good to have:
  • Experience with custom ETL/data pipeline design
  • Experience troubleshooting large-scale systems
  • Understanding of Hadoop, Spark, Pig, Hive
  • Good communication skills
  • Agile development experience

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. About the Department Video and Edge is one of the world's leading video platforms that provides media storage, delivery, transcoding, and streaming services. We are building the next generation video processing platform and the largest live streaming network, which provides excellent experiences for billions of users around the world. Popular video products of TikTok and its affiliates are all empowered by our cutting-edge cloud technologies. Working in this team, you will have the opportunity to tackle challenges of large-scale networks all over the world, while leveraging your expertise in coding, algorithms, complexity analysis, and large-scale system design. About the Team The Data Warehouse team within Video Architecture provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges. Responsibilities - Design and develop optimal data processing architecture and systems for new data and ETL (Extract, Transform, Load) pipelines. - Architect, implement, and maintain efficient and reliable data pipelines for data movement and transformation across varying scales. - Drive internal process improvements and automate manual processes to enhance data quality and Service Level Agreement (SLA) management. - Collaborate with cross-functional teams including Content Delivery Network (CDN), Video Understanding, Video Transcoding, Live Streaming, and Real-Time Communication teams.
Qualifications
Minimum Qualifications - Bachelor's degree in Computer Science or a related technical field involving software/system engineering, or equivalent working experience. - Good programming experience with at least one of the following languages: Spark SQL, Python, NoSQL databases. - Good understanding of data model design and data warehouse architecture theories. Preferred Qualifications - Experience in custom ETL/data pipeline design, implementation, and maintenance. - Experience in troubleshooting large-scale distributed systems. - Good understanding of big-data processing software (Hadoop, Spark, Pig, Hive) and algorithms (MapReduce). - Good communication skills and willingness to work closely with peer teams for successful deliveries. - Willingness to embrace agile development for rapid iteration. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

Microsoft - Member of Technical Staff, AI - Multimodal

Microsoft

(On-Site)
• 1 Day ago
Warner Bros Discovery - Customer Data Manager - Digital/ VOD

Warner Bros Discovery

Masovian Voivodeship, Poland (Hybrid)
• 3 Months ago
Dream Games - Senior Graphic Designer

Dream Games

İstanbul, Türkiye (On-Site)
• 6 Months ago
ByteDance - Large Language Model Algorithm Engineer - Volcano Ark

ByteDance

Singapore (On-Site)
• 3 Months ago
Meta - Software Engineer, Computer Vision (Technical Leadership)

Meta

Burlingame, California, United States (Remote)
• 3 Months ago
GoTo Group - Senior Data Warehouse Engineer - India

GoTo Group

Bengaluru, Karnataka, India (On-Site)
• 4 Months ago
ION - Internship - Data Science

ION

Milan, Lombardy, Italy (On-Site)
• 4 Months ago
Playtika - Experienced Business Analyst

Playtika

Israel (On-Site)
• 3 Months ago
GoTo Group - Senior Data Scientist (Driver Risk)

GoTo Group

Jakarta, Jakarta, Indonesia (On-Site)
• 2 Months ago
Porch Group - Sr. Data Scientist

Porch Group

India (Remote)
• 3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Google - Software Engineer III, Google Assistant

Google

Bengaluru, Karnataka, India (On-Site)
• 3 Months ago
NVIDIA - Deep Learning Engineer, Generative AI and 3D Reconstruction

NVIDIA

Tokyo, Japan (On-Site)
• 3 Weeks ago
PhonePe - Firmware Engineer (Exp. Bucket 3-5 Yrs)

PhonePe

Bengaluru, Karnataka, India (On-Site)
• 3 Months ago
Wildlife Studios - Data Engineer

Wildlife Studios

São Paulo, State Of São Paulo, Brazil (On-Site)
• 1 Month ago
Tesla - Distributed Systems Engineer, Autobidder Platform

Tesla

North Holland, Netherlands (On-Site)
• 1 Week ago
Meta - Research Scientist Intern, Language and Multimodal Research for MetaAI (PhD)

Meta

Menlo Park, California, United States (On-Site)
• 3 Months ago
Zoox - Director of Perception

Zoox

Foster City, California, United States (Hybrid)
• 4 Months ago
Argus Labs - Software Engineer (Infrastructure/Backend)

Argus Labs

(Remote)
• 3 Weeks ago
Warner Bros Games - Software Engineer II - Backend

Warner Bros Games

Bengaluru, Karnataka, India (Hybrid)
• 5 Days ago
The Walt Disney Company - Ride Controls Systems Engineer

The Walt Disney Company

Glendale, California, United States (On-Site)
• 5 Days ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

OKX - Leadership Growth Expert, Product & Engineering

OKX

Singapore, Singapore (On-Site)
• 4 Months ago
ION - Service Desk Analyst - 5600

ION

Singapore (On-Site)
• 4 Months ago
Netflix - Operations Import Manager - APAC

Netflix

Singapore, Singapore (On-Site)
• 1 Month ago
ByteDance - Backend Engineer (Model Inference), Machine Learning Systems

ByteDance

Singapore (On-Site)
• 3 Months ago
ByteDance - Production System Engineer, Infrastructure Engineering

ByteDance

Singapore (On-Site)
• 3 Months ago
PwC - Workforce Transformation Manager

PwC

Singapore (On-Site)
• 4 Months ago
ByteDance - DevOps Engineer - Applied Machine Learning Engine (Singapore)

ByteDance

Singapore (On-Site)
• 3 Months ago
IGG - Senior 2D Concept Artist

IGG

Singapore (On-Site)
• 3 Months ago
SSC Technologies - Associate Manager, Hedge Funds

SSC Technologies

Singapore (On-Site)
• 4 Months ago
HoYoverse - Brand Marketing - Fresh Grad

HoYoverse

Singapore (On-Site)
• 6 Days ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

Tencent - Site Reliability Engineer

Tencent

(On-Site)
• 1 Month ago
Social Discovery Group - Senior Product Analyst (Virtual Gifts)

Social Discovery Group

Tbilisi, Tbilisi, Georgia (Remote)
• 1 Month ago
Corsair - BI Developer

Corsair

Vietnam (On-Site)
• 1 Month ago
Eneba Games - Business Intelligence Analyst, Payments

Eneba Games

Lithuania (Remote)
• 2 Days ago
Porch Group - Sr. Data Scientist

Porch Group

India (Remote)
• 3 Months ago
GoMotive - Senior Strategic Analyst, Product Reliability

GoMotive

United States (Remote)
• 6 Days ago
NVIDIA - Senior Account Manager, Global Data Center and NCP Strategies

NVIDIA

Santa Clara, California, United States (On-Site)
• 1 Month ago
Nintendo - Associate Insights and Analytics Analyst

Nintendo

Redmond, Washington, United States (Hybrid)
• 2 Months ago
GoTo Group - Senior Growth Analyst - Gopay Pinjam

GoTo Group

Jakarta, Jakarta, Indonesia (On-Site)
• 3 Months ago
Rackspace Technology - R-19462 Data Engineer III - VN

Rackspace Technology

Vietnam (Remote)
• 1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Where imagination meets innovation, delivering limitless gaming experiences.

Taguig, Metro Manila, Philippines (On-Site)

Singapore (On-Site)

Dubai, Dubai, United Arab Emirates (On-Site)

State Of São Paulo, Brazil (On-Site)

Seattle, Washington, United States (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by ByteDance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug