Data Engineer - Video Architecture, LiveNet (Live Streaming Networks)

4 Months ago • All levels • Data Analysis

Job Summary

Job Description

ByteDance's Video and Edge department seeks a Data Engineer for its LiveNet (Live Streaming Networks) team. Responsibilities include designing and developing data processing architectures and ETL pipelines, architecting and maintaining efficient data pipelines, improving internal processes, and collaborating with cross-functional teams (CDN, Video Understanding, etc.). The role involves working with big data infrastructure and platforms to provide high-quality data to various teams, supporting core business metrics analysis. The ideal candidate possesses strong programming skills (Spark SQL, Python, NoSQL), understands data modeling and warehouse architecture, and has experience with big data processing software (Hadoop, Spark).
Must have:
  • Design and develop data processing architecture and systems
  • Build efficient and reliable data pipelines
  • Improve data quality and SLA management
  • Collaborate with cross-functional teams
  • Experience with Spark SQL, Python, or NoSQL databases
  • Understanding of data model design and data warehouse architecture
Good to have:
  • Experience with custom ETL/data pipeline design
  • Experience troubleshooting large-scale systems
  • Understanding of Hadoop, Spark, Pig, Hive
  • Good communication skills
  • Agile development experience

Job Details

Responsibilities
About ByteDance Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. Why Join Us Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always. At ByteDance, we create together and grow together. That's how we drive impact-for ourselves, our company, and the users we serve. Join us. About the Department Video and Edge is one of the world's leading video platforms that provides media storage, delivery, transcoding, and streaming services. We are building the next generation video processing platform and the largest live streaming network, which provides excellent experiences for billions of users around the world. Popular video products of TikTok and its affiliates are all empowered by our cutting-edge cloud technologies. Working in this team, you will have the opportunity to tackle challenges of large-scale networks all over the world, while leveraging your expertise in coding, algorithms, complexity analysis, and large-scale system design. About the Team The Data Warehouse team within Video Architecture provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges. Responsibilities - Design and develop optimal data processing architecture and systems for new data and ETL (Extract, Transform, Load) pipelines. - Architect, implement, and maintain efficient and reliable data pipelines for data movement and transformation across varying scales. - Drive internal process improvements and automate manual processes to enhance data quality and Service Level Agreement (SLA) management. - Collaborate with cross-functional teams including Content Delivery Network (CDN), Video Understanding, Video Transcoding, Live Streaming, and Real-Time Communication teams.
Qualifications
Minimum Qualifications - Bachelor's degree in Computer Science or a related technical field involving software/system engineering, or equivalent working experience. - Good programming experience with at least one of the following languages: Spark SQL, Python, NoSQL databases. - Good understanding of data model design and data warehouse architecture theories. Preferred Qualifications - Experience in custom ETL/data pipeline design, implementation, and maintenance. - Experience in troubleshooting large-scale distributed systems. - Good understanding of big-data processing software (Hadoop, Spark, Pig, Hive) and algorithms (MapReduce). - Good communication skills and willingness to work closely with peer teams for successful deliveries. - Willingness to embrace agile development for rapid iteration. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Similar Jobs

LeoVegas - Senior Market Researcher

LeoVegas

Leeds, England, United Kingdom (Hybrid)
6 Days ago
Workato - Manager, FP&A

Workato

Palo Alto, California, United States (On-Site)
1 Month ago
Sporty - CS Saas Product Manager

Sporty

(Remote)
1 Year ago
Mythical Games - Senior User Acquisition Manager

Mythical Games

United States (Remote)
1 Month ago
Unity - Account Manager

Unity

Seoul, South Korea (On-Site)
1 Month ago
ShyftLabs - Staff Data Engineer

ShyftLabs

Toronto, Ontario, Canada (Hybrid)
1 Month ago
Haptic  - Lead Data Scientist and User Researcher

Haptic

Germany (On-Site)
6 Months ago
Flexra Software - Staff Data Engineer

Flexra Software

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Spyke Games - Data Analyst

Spyke Games

İstanbul, Türkiye (On-Site)
1 Year ago
miniclip - Senior Data Analyst

miniclip

London, England, United Kingdom (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

techholding - Varicent ICM Engineer

techholding

Gurugram, Haryana, India (On-Site)
1 Month ago
bytedance - Hardware Product Engineer (VR/AR)- PICO

bytedance

San Jose, California, United States (On-Site)
8 Months ago
Illumina - Sr. Product Marketing Manager, NGS Platforms

Illumina

San Diego, California, United States (On-Site)
1 Month ago
Peak Games - HR Specialist

Peak Games

(On-Site)
4 Months ago
Canva - Senior HarmonyOS + Android Engineer

Canva

Beijing, China (Remote)
1 Month ago
Sprinkler - Technical Support Engineer

Sprinkler

Bengaluru, Karnataka, India (On-Site)
1 Month ago
Epic Games - Animator

Epic Games

Manchester, England, United Kingdom (On-Site)
1 Month ago
Clearwater Analytics - Professional Services Leader

Clearwater Analytics

Boise, Idaho, United States (On-Site)
3 Weeks ago
Greenworks Sunrise Global Marketing - Sr. Product Manager, Handheld

Greenworks Sunrise Global Marketing

Mooresville, North Carolina, United States (On-Site)
1 Month ago
Veeam Software - Director, Marketing AI Strategist

Veeam Software

Florida, United States (Remote)
2 Weeks ago

Get notifed when new similar jobs are uploaded

Jobs in Singapore

bytedance - Cloud Technical Support

bytedance

Singapore (On-Site)
3 Months ago
Sailpoint - Digital Sales Representative

Sailpoint

Singapore (Hybrid)
1 Month ago
Interactive Brokers - Application Service Group Associate

Interactive Brokers

Singapore (On-Site)
3 Weeks ago
bytedance - Service Framework Software Engineer Intern

bytedance

Singapore (On-Site)
3 Months ago
NinjaVan - Air Freight Sales Manager

NinjaVan

Singapore (On-Site)
2 Weeks ago
Riot Games - Motion Graphics Artist

Riot Games

Singapore (On-Site)
1 Month ago
Mcdonalds - Director of Global Supply Chain Field Service

Mcdonalds

Singapore (On-Site)
2 Weeks ago
Tencent - Global Talent Sourcing Intern

Tencent

Singapore (On-Site)
9 Months ago
Thales - Software Engineer (Embedded Systems)

Thales

Singapore, Singapore (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

Data Analysis Jobs

Moloco - Senior Data Scientist, Growth Analytics

Moloco

Tokyo, Japan (On-Site)
1 Month ago
Zuora - Business Systems Analyst

Zuora

Redwood City, California, United States (Remote)
1 Month ago
MURKA - Marketing Data Analyst

MURKA

Poland (On-Site)
5 Months ago
YouGov - Senior Data Scientist

YouGov

United States (Remote)
3 Weeks ago
Highspot - Sr. Data Engineer

Highspot

Hyderabad, Telangana, India (Hybrid)
4 Months ago
bytedance - Senior Research Scientist, Data Management and Security - Infrastructure System Lab

bytedance

San Jose, California, United States (On-Site)
3 Months ago
Tencent - Marketing Data Analyst (深圳)

Tencent

Shenzhen, Guangdong Province, China (On-Site)
8 Months ago
CommerceIQ - Business Analyst II

CommerceIQ

Bengaluru, Karnataka, India (On-Site)
2 Months ago
oportun - Senior Data Scientist

oportun

(Remote)
2 Months ago
Minecast - Senior Data Engineer (NLP & ML)

Minecast

London, England, United Kingdom (Hybrid)
1 Month ago

Get notifed when new similar jobs are uploaded

About The Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

San Jose, California, United States (On-Site)

San Jose, California, United States (On-Site)

Seattle, Washington, United States (On-Site)

Seattle, Washington, United States (On-Site)

San Jose, California, United States (On-Site)

View All Jobs

Get notified when new jobs are added by bytedance

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug