Data Engineer - Video Architecture, LiveNet (Live Streaming Networks)

5 Months ago • All levels • Data Analysis

Job Summary

Job Description

ByteDance's Video and Edge department seeks a Data Engineer for its LiveNet (Live Streaming Networks) team. Responsibilities include designing and developing data processing architectures and ETL pipelines, architecting and maintaining efficient data pipelines, improving internal processes, and collaborating with cross-functional teams (CDN, Video Understanding, etc.). The role involves working with big data infrastructure and platforms to provide high-quality data to various teams, supporting core business metrics analysis. The ideal candidate possesses strong programming skills (Spark SQL, Python, NoSQL), understands data modeling and warehouse architecture, and has experience with big data processing software (Hadoop, Spark).
Must have:
  • Design and develop data processing architecture and systems
  • Build efficient and reliable data pipelines
  • Improve data quality and SLA management
  • Collaborate with cross-functional teams
  • Experience with Spark SQL, Python, or NoSQL databases
  • Understanding of data model design and data warehouse architecture
Good to have:
  • Experience with custom ETL/data pipeline design
  • Experience troubleshooting large-scale systems
  • Understanding of Hadoop, Spark, Pig, Hive
  • Good communication skills
  • Agile development experience

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. About the Department Video and Edge is one of the world's leading video platforms that provides media storage, delivery, transcoding, and streaming services. We are building the next generation video processing platform and the largest live streaming network, which provides excellent experiences for billions of users around the world. Popular video products of TikTok and its affiliates are all empowered by our cutting-edge cloud technologies. Working in this team, you will have the opportunity to tackle challenges of large-scale networks all over the world, while leveraging your expertise in coding, algorithms, complexity analysis, and large-scale system design. About the Team The Data Warehouse team within Video Architecture provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges. Responsibilities - Design and develop optimal data processing architecture and systems for new data and ETL (Extract, Transform, Load) pipelines. - Architect, implement, and maintain efficient and reliable data pipelines for data movement and transformation across varying scales. - Drive internal process improvements and automate manual processes to enhance data quality and Service Level Agreement (SLA) management. - Collaborate with cross-functional teams including Content Delivery Network (CDN), Video Understanding, Video Transcoding, Live Streaming, and Real-Time Communication teams.
Qualifications
Minimum Qualifications - Bachelor's degree in Computer Science or a related technical field involving software/system engineering, or equivalent working experience. - Good programming experience with at least one of the following languages: Spark SQL, Python, NoSQL databases. - Good understanding of data model design and data warehouse architecture theories. Preferred Qualifications - Experience in custom ETL/data pipeline design, implementation, and maintenance. - Experience in troubleshooting large-scale distributed systems. - Good understanding of big-data processing software (Hadoop, Spark, Pig, Hive) and algorithms (MapReduce). - Good communication skills and willingness to work closely with peer teams for successful deliveries. - Willingness to embrace agile development for rapid iteration. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

JDA - Senior End-to-End Solution Advisor - Retail

JDA

London, England, United Kingdom (On-Site)
1 Year ago
Zuora - Senior Product Manager, Billing

Zuora

Redwood City, California, United States (On-Site)
1 Month ago
LeoVegas - Commercial Marketing Manager - Retention

LeoVegas

Newcastle Upon Tyne, England, United Kingdom (On-Site)
3 Weeks ago
Epic Games - Product Manager (BR & Reload)

Epic Games

(On-Site)
6 Months ago
Riot Games - Senior Manager, Game Product Management - Unpublished R&D Product

Riot Games

Los Angeles, California, United States (On-Site)
3 Months ago
Apple - Data Engineering Software Engineer

Apple

San Francisco, California, United States (On-Site)
2 Months ago
Colo pl - Data Scientist / Analyst

Colo pl

Tokyo, Japan (On-Site)
3 Months ago
endava - Senior Data Engineer - GCP

endava

Sydney, New South Wales, Australia (On-Site)
1 Month ago
Apple - AIML - Senior Data Scientist, Evaluation

Apple

Seattle, Washington, United States (On-Site)
4 Weeks ago
Apple - Software Engineer (Data), AI & Data Platforms

Apple

Sunnyvale, California, United States (On-Site)
4 Weeks ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Sierra - Partner Manager, System Integrator

Sierra

San Francisco, California, United States (On-Site)
2 Months ago
HappyRobot - Growth Engineer

HappyRobot

San Francisco, California, United States (On-Site)
1 Week ago
Rackspace Technology - Operational Lead – Day-to-Day Operations

Rackspace Technology

Riyadh, Riyadh Province, Saudi Arabia (On-Site)
1 Month ago
IMC - Operational Delivery Manager

IMC

Amsterdam, North Holland, Netherlands (On-Site)
2 Months ago
Visa - Director, E-commerce Platforms

Visa

Atlanta, Georgia, United States (Hybrid)
2 Months ago
Philips - Sales, Key Account Manager - Hospital Patient Monitoring

Philips

Las Vegas, Nevada, United States (On-Site)
2 Months ago
Palo Alto Networks - Senior Principal DevSecOps Engineer

Palo Alto Networks

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
1 Week ago
Epic Games - Director, Marketing

Epic Games

Cary, North Carolina, United States (On-Site)
3 Months ago
GLu Mobile - Associate Product Manager, Live Ops

GLu Mobile

Vancouver, British Columbia, Canada (Hybrid)
4 Weeks ago
Paytm - Technical Program Manager

Paytm

Bengaluru, Karnataka, India (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

GoTo Group - Sales Associate - GoCorp

GoTo Group

Singapore (Hybrid)
6 Days ago
Windranger - Blockchain Engineer

Windranger

Singapore (Remote)
2 Years ago
bytedance - Senior Product Manager - Cloud Security

bytedance

Singapore (On-Site)
8 Months ago
HoYoverse - Accountant (Accounts Payable)

HoYoverse

Singapore (On-Site)
3 Months ago
Ubisoft - Data Director

Ubisoft

Singapore, Singapore (On-Site)
4 Months ago
Riot Games - Senior Service Designer, Player Support

Riot Games

Singapore (On-Site)
4 Months ago
Workato - Commercial Account Executive

Workato

Singapore (On-Site)
4 Weeks ago
Marvell - Design Validation Engineer Intern - Masters

Marvell

Singapore (On-Site)
1 Month ago
Philips - Regional Planner APAC

Philips

Singapore (On-Site)
6 Days ago
Motorola solutions - Regional Account Manager (Asia)

Motorola solutions

Singapore (On-Site)
1 Year ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Next Level Business Services - Data Architect

Next Level Business Services

Sunnyvale, California, United States (On-Site)
9 Months ago
Scopely - Senior Data Analyst, Marketing Analytics

Scopely

Mexico City, Mexico City, Mexico (Hybrid)
5 Months ago
MPOWER Financing - Data Engineer - Data - Bangalore, India

MPOWER Financing

Bengaluru, Karnataka, India (On-Site)
8 Months ago
Sony Pictures Entertainment - SVP, Data & Analytics

Sony Pictures Entertainment

Culver City, California, United States (Hybrid)
2 Months ago
Siemens  - Data Integrity Analyst

Siemens

Noida, Uttar Pradesh, India (On-Site)
3 Weeks ago
Unity - Software Engineer - Big Data & Backend

Unity

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
3 Months ago
Thumbtack - Senior Data Engineer

Thumbtack

United States (Remote)
1 Month ago
Extreme Inc. - Data Analysis Engineer

Extreme Inc.

Tokyo, Japan (Hybrid)
3 Months ago
London stock Exchange - Data Lineage Engineer

London stock Exchange

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Zinnia - Revenue Operations Manager (Data and Analytics)

Zinnia

New York, New York, United States (Hybrid)
2 Weeks ago

Get notifed when new similar jobs are uploaded

About The Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.
View All Jobs

Get notified when new jobs are added by bytedance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug