Data Engineer - Video Architecture, LiveNet (Live Streaming Networks)

2 Months ago • All levels • Data Analyst

Job Summary

Job Description

ByteDance's Video and Edge department seeks a Data Engineer for its LiveNet (Live Streaming Networks) team. Responsibilities include designing and developing data processing architectures and ETL pipelines, architecting and maintaining efficient data pipelines, improving internal processes, and collaborating with cross-functional teams (CDN, Video Understanding, etc.). The role involves working with big data infrastructure and platforms to provide high-quality data to various teams, supporting core business metrics analysis. The ideal candidate possesses strong programming skills (Spark SQL, Python, NoSQL), understands data modeling and warehouse architecture, and has experience with big data processing software (Hadoop, Spark).
Must have:
  • Design and develop data processing architecture and systems
  • Build efficient and reliable data pipelines
  • Improve data quality and SLA management
  • Collaborate with cross-functional teams
  • Experience with Spark SQL, Python, or NoSQL databases
  • Understanding of data model design and data warehouse architecture
Good to have:
  • Experience with custom ETL/data pipeline design
  • Experience troubleshooting large-scale systems
  • Understanding of Hadoop, Spark, Pig, Hive
  • Good communication skills
  • Agile development experience

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. About the Department Video and Edge is one of the world's leading video platforms that provides media storage, delivery, transcoding, and streaming services. We are building the next generation video processing platform and the largest live streaming network, which provides excellent experiences for billions of users around the world. Popular video products of TikTok and its affiliates are all empowered by our cutting-edge cloud technologies. Working in this team, you will have the opportunity to tackle challenges of large-scale networks all over the world, while leveraging your expertise in coding, algorithms, complexity analysis, and large-scale system design. About the Team The Data Warehouse team within Video Architecture provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges. Responsibilities - Design and develop optimal data processing architecture and systems for new data and ETL (Extract, Transform, Load) pipelines. - Architect, implement, and maintain efficient and reliable data pipelines for data movement and transformation across varying scales. - Drive internal process improvements and automate manual processes to enhance data quality and Service Level Agreement (SLA) management. - Collaborate with cross-functional teams including Content Delivery Network (CDN), Video Understanding, Video Transcoding, Live Streaming, and Real-Time Communication teams.
Qualifications
Minimum Qualifications - Bachelor's degree in Computer Science or a related technical field involving software/system engineering, or equivalent working experience. - Good programming experience with at least one of the following languages: Spark SQL, Python, NoSQL databases. - Good understanding of data model design and data warehouse architecture theories. Preferred Qualifications - Experience in custom ETL/data pipeline design, implementation, and maintenance. - Experience in troubleshooting large-scale distributed systems. - Good understanding of big-data processing software (Hadoop, Spark, Pig, Hive) and algorithms (MapReduce). - Good communication skills and willingness to work closely with peer teams for successful deliveries. - Willingness to embrace agile development for rapid iteration. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

NVIDIA - Senior Circuit Design Engineer

NVIDIA

Austin, Texas, United States (Hybrid)
2 Weeks ago
Whoop - Senior Sensor Algorithm Engineer

Whoop

Boston, Massachusetts, United States (On-Site)
22 Hours ago
QuinStreet - Entry Level Sales Representative

QuinStreet

Orlando, Florida, United States (Hybrid)
1 Day ago
Mozilla - Staff Machine Learning Engineer, Gen AI

Mozilla

Netherlands (Remote)
6 Months ago
Hitachi - Data Scientist

Hitachi

Pune, Maharashtra, India (Remote)
6 Months ago
Jam City - Product Manager

Jam City

Buenos Aires, Buenos Aires, Argentina (On-Site)
8 Months ago
Ubisoft - Legal Counsel Data Protection Assistant - Internship

Ubisoft

Paris, Île-de-France, France (On-Site)
1 Month ago
PwC - IGP_CP&F - Data Governance  - Senior Associate - Bangalore

PwC

Bengaluru, Karnataka, India (On-Site)
7 Months ago
Equivalent Jobs - ML ENGINEER

Equivalent Jobs

(Remote)
5 Months ago
Easygo - Data Scientist

Easygo

Melbourne, Victoria, Australia (On-Site)
3 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Google - Software Engineer, Google Play, Subscription Growth

Google

Atlanta, Georgia, United States (On-Site)
2 Weeks ago
Google - Software Engineer, Early Career, Campus

Google

Warsaw, Masovian Voivodeship, Poland (On-Site)
2 Weeks ago
Zscaler - Staff Machine Learning Engineer

Zscaler

Bengaluru, Karnataka, India (Hybrid)
7 Hours ago
Ansys - Lead R&D Engineer - C++, HPC for EDA Simulation

Ansys

Athens, Greece (On-Site)
3 Days ago
Opendoor - Creative Marketing Manager, Organic Social Media

Opendoor

San Francisco, California, United States (Hybrid)
6 Hours ago
SLAY - Senior Marketing Data Engineer (SKAN Attribution, LTV forecasting)

SLAY

Berlin, Berlin, Germany (On-Site)
1 Month ago
Google - Software Engineer II, iOS Developer

Google

Tel Aviv-Yafo, Tel Aviv District, Israel (On-Site)
1 Week ago
Fictiv - Associate TPM I

Fictiv

Guangzhou, Guangdong Province, China (On-Site)
7 Hours ago
Equivalent Jobs - C++ SOFTWARE ENGINEER (SIMULATOR)

Equivalent Jobs

(Remote)
5 Months ago
Google - Software Developer III, Site Reliability Development

Google

Waterloo, Ontario, Canada (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

ByteDance - Contract Management Specialist

ByteDance

Singapore (On-Site)
2 Weeks ago
Nium - Staff Solution Engineer

Nium

Singapore, Singapore (Hybrid)
3 Weeks ago
OLIVER Agency - GenAI Motion Graphic Designer

OLIVER Agency

Singapore (On-Site)
21 Hours ago
ByteDance - Production System Engineer, Infrastructure Engineering

ByteDance

Singapore (On-Site)
6 Months ago
Ambition - Data Engineer (python)

Ambition

Singapore (On-Site)
8 Months ago
ByteDance - Security Engineer (Penetration Tester) - 2025 Start

ByteDance

Singapore (On-Site)
6 Months ago
ByteDance - Service Framework Software Engineer Intern

ByteDance

Singapore (On-Site)
1 Month ago
ByteDance - Security Software Engineer

ByteDance

Singapore (On-Site)
2 Weeks ago
ByteDance - DevOps Engineer - Applied Machine Learning Engine (Singapore)

ByteDance

Singapore (On-Site)
5 Months ago

Get notifed when new similar jobs are uploaded

Data Analyst Jobs

PwC - IN-Senior Associate_Power BI_Data & Analytics_Advisory_Mumbai

PwC

Mumbai, Maharashtra, India (On-Site)
6 Months ago
ByteDance - Data Analyst - Global Payment

ByteDance

Singapore (On-Site)
1 Month ago
Virtuos - R&D Data Engineering Lead

Virtuos

Shanghai, Shanghai, China (On-Site)
1 Month ago
Canva - Senior Analytics Engineer

Canva

Sydney, New South Wales, Australia (Remote)
1 Week ago
Google - Analytical Lead, Large Customer Sales

Google

Buenos Aires, Buenos Aires, Argentina (On-Site)
2 Weeks ago
DraftKings - Manager, Risk Analytics

DraftKings

Boston, Massachusetts, United States (Hybrid)
1 Month ago
PwC - Senior BI/Data analyst – Technology Consulting

PwC

Prague, Prague, Czechia (On-Site)
6 Months ago
THE GAME - Data & Insights Analyst

THE GAME

London, England, United Kingdom (Hybrid)
1 Month ago
N-iX - Senior Manual QA (With Data Experience)

N-iX

Ukraine (Remote)
2 Weeks ago
The Walt Disney Company - Senior Data Engineer

The Walt Disney Company

Glendale, California, United States (On-Site)
1 Week ago

Get notifed when new similar jobs are uploaded

About The Company

Where imagination meets innovation, delivering limitless gaming experiences.

San Diego, California, United States (On-Site)

San Jose, California, United States (On-Site)

Dubai, Dubai, United Arab Emirates (On-Site)

New York, New York, United States (On-Site)

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

Seattle, Washington, United States (On-Site)

View All Jobs

Get notified when new jobs are added by ByteDance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug