Data Engineer - Video Architecture, LiveNet (Live Streaming Networks)

2 Months ago • All levels • Data Analyst

Job Summary

Job Description

ByteDance's Video and Edge department seeks a Data Engineer for its LiveNet (Live Streaming Networks) team. Responsibilities include designing and developing data processing architectures and ETL pipelines, architecting and maintaining efficient data pipelines, improving internal processes, and collaborating with cross-functional teams (CDN, Video Understanding, etc.). The role involves working with big data infrastructure and platforms to provide high-quality data to various teams, supporting core business metrics analysis. The ideal candidate possesses strong programming skills (Spark SQL, Python, NoSQL), understands data modeling and warehouse architecture, and has experience with big data processing software (Hadoop, Spark).
Must have:
  • Design and develop data processing architecture and systems
  • Build efficient and reliable data pipelines
  • Improve data quality and SLA management
  • Collaborate with cross-functional teams
  • Experience with Spark SQL, Python, or NoSQL databases
  • Understanding of data model design and data warehouse architecture
Good to have:
  • Experience with custom ETL/data pipeline design
  • Experience troubleshooting large-scale systems
  • Understanding of Hadoop, Spark, Pig, Hive
  • Good communication skills
  • Agile development experience

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. About the Department Video and Edge is one of the world's leading video platforms that provides media storage, delivery, transcoding, and streaming services. We are building the next generation video processing platform and the largest live streaming network, which provides excellent experiences for billions of users around the world. Popular video products of TikTok and its affiliates are all empowered by our cutting-edge cloud technologies. Working in this team, you will have the opportunity to tackle challenges of large-scale networks all over the world, while leveraging your expertise in coding, algorithms, complexity analysis, and large-scale system design. About the Team The Data Warehouse team within Video Architecture provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges. Responsibilities - Design and develop optimal data processing architecture and systems for new data and ETL (Extract, Transform, Load) pipelines. - Architect, implement, and maintain efficient and reliable data pipelines for data movement and transformation across varying scales. - Drive internal process improvements and automate manual processes to enhance data quality and Service Level Agreement (SLA) management. - Collaborate with cross-functional teams including Content Delivery Network (CDN), Video Understanding, Video Transcoding, Live Streaming, and Real-Time Communication teams.
Qualifications
Minimum Qualifications - Bachelor's degree in Computer Science or a related technical field involving software/system engineering, or equivalent working experience. - Good programming experience with at least one of the following languages: Spark SQL, Python, NoSQL databases. - Good understanding of data model design and data warehouse architecture theories. Preferred Qualifications - Experience in custom ETL/data pipeline design, implementation, and maintenance. - Experience in troubleshooting large-scale distributed systems. - Good understanding of big-data processing software (Hadoop, Spark, Pig, Hive) and algorithms (MapReduce). - Good communication skills and willingness to work closely with peer teams for successful deliveries. - Willingness to embrace agile development for rapid iteration. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

ByteDance - Software Development Engineer (SDN Traffic Intelligence & Control)

ByteDance

San Jose, California, United States (On-Site)
4 Weeks ago
Token Metrics - Crypto Data Scientist / Machine Learning Engineer  (Remote)

Token Metrics

Tiranë, Tirana County, Albania (Remote)
6 Months ago
Google - Staff Software Engineer, Google Cloud

Google

Bengaluru, Karnataka, India (On-Site)
4 Months ago
Google - Staff Software Engineer, Engineering Productivity, Google Cloud Platforms

Google

Sunnyvale, California, United States (On-Site)
1 Week ago
Hitachi - Data Scientist

Hitachi

Pune, Maharashtra, India (Remote)
6 Months ago
Warner Bros Games - Senior Software Engineer

Warner Bros Games

New York, New York, United States (Hybrid)
4 Weeks ago
Google - Fabrication Data and Yield Hardware Engineer

Google

Goleta, California, United States (On-Site)
1 Week ago
Applike Group - Senior Data Scientist (Recommendation Systems Expert) (f/m/d)

Applike Group

Hamburg, Hamburg, Germany (Hybrid)
6 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Booming games - Game Integrity Manager

Booming games

(Remote)
2 Months ago
Google - Software Engineer III, Infrastructure

Google

Mountain View, California, United States (On-Site)
1 Week ago
Google - Senior Software Engineer, Mobile (Android), Google Workspace

Google

Sunnyvale, California, United States (On-Site)
1 Week ago
Google - Software Engineer III, Full Stack, Google Ads

Google

Mountain View, California, United States (On-Site)
5 Months ago
Netflix - Machine Learning Manager - Promotional Media

Netflix

United States (Remote)
1 Week ago
Google - Software Engineer II, Incident Response Management, Cloud

Google

London, England, United Kingdom (On-Site)
1 Week ago
Epic Games - Research Engineer - Neural Rendering

Epic Games

London, England, United Kingdom (On-Site)
1 Week ago
Epic Games - Senior Physics Programmer

Epic Games

Cary, North Carolina, United States (On-Site)
3 Months ago
NVIDIA - Machine Learning Intern - 2025

NVIDIA

(On-Site)
2 Months ago
Google - Software Engineer II, Service Provisioning, Google Cloud Platform

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

PwC - Workforce Transformation Senior Manager

PwC

Singapore (On-Site)
6 Months ago
ByteDance - Data Analyst - Corporate Information System

ByteDance

Singapore (On-Site)
1 Month ago
ByteDance - DevOps Engineer - Applied Machine Learning Engine (Singapore)

ByteDance

Singapore (On-Site)
5 Months ago
Razer - Associate Director, Software Product Marketing

Razer

Singapore (On-Site)
6 Months ago
Google - Senior Data Center Strategic Negotiator, ELS Regional Development

Google

Singapore, Singapore (On-Site)
1 Week ago
ByteDance - Ad Delivery Algorithm Engineer-Game

ByteDance

Singapore (On-Site)
5 Months ago
NinjaVan - Class 3 Delivery Staff Driver (Full Time, 10 Footer Lorry)

NinjaVan

Singapore, Singapore (On-Site)
6 Months ago
Google - Social Insight Strategist

Google

Singapore (On-Site)
6 Days ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

Ourpalm - Senior Overseas Advertising Optimization Specialist

Ourpalm

Beijing, Beijing, China (On-Site)
3 Months ago
Electronic Arts - Fraud Analyst

Electronic Arts

Austin, Texas, United States (On-Site)
1 Week ago
Info Stretch - Programmer Analyst 6

Info Stretch

Lansing, Michigan, United States (Hybrid)
4 Months ago
Playrix - Senior Data Analyst (Game)

Playrix

Armenia (Remote)
6 Months ago
GT - Data Governance Lead

GT

Poland (Remote)
4 Weeks ago
Chess - Senior Data Scientist - Product

Chess

United States (Remote)
10 Months ago
Wildlife Studios - IT Cloud Governance Analyst

Wildlife Studios

São Paulo, State Of São Paulo, Brazil (On-Site)
4 Months ago
ByteDance - Data Engineer, Cloud and System

ByteDance

Seattle, Washington, United States (On-Site)
1 Month ago
Aristocrat Gaming - Data Owner & Analyst - ERP D365

Aristocrat Gaming

Tel Aviv-Yafo, Tel Aviv District, Israel (Hybrid)
2 Weeks ago
ION - Data Associate - KYC6

ION

Budapest, Hungary (On-Site)
6 Months ago

Get notifed when new similar jobs are uploaded

About The Company

Where imagination meets innovation, delivering limitless gaming experiences.

Dublin, County Dublin, Ireland (On-Site)

London, England, United Kingdom (On-Site)

Bangkok, Bangkok, Thailand (On-Site)

San Jose, California, United States (On-Site)

State Of São Paulo, Brazil (On-Site)

San Jose, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by ByteDance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug