Data Engineer - TikTok, Video-on-Demand

Description

Team Introduction
The Data Warehouse team within Video-on-Demand provides stable, complete, and high-quality data to DA/DS/RD/PM teams in a cost-effective manner. Our work includes building data pipelines, optimizing data workflows, and tackling other big data challenges using leading big-data infrastructure and platforms. The team's main goal is to help internal teams and stakeholders gain deep insights into their core business metrics, including service costs and quality metrics. Working on this team, you'll collaborate with one of the largest network system teams to build advanced data models and solve sophisticated data challenges.

Responsibilities:
- Design and build resilient and efficient data pipelines for both batch and real-time streaming workloads.
- Develop end-to-end data solutions, from data ingestion and processing to data persistence and service layer development.
- Maintain and improve existing pipelines for better scalability, adaptability, and maintainability.
- Collaborate with data scientists, analysts, product managers, and various engineering teams.
- Engineer scalable solutions for both structured and unstructured data.
- Continuously identify and test internal/external opportunities to optimize product and service performance through data.

Requirements

Minimum Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience).
- 4+ years of hands-on experience working primarily with data in roles such as Data Engineer, Data Analyst, or Data Scientist.
- Proficient in SQL, data modeling, ETL pipeline development, and at least one programming language (e.g., Python, Java, Go, or Scala).
- Strong experience with distributed data processing frameworks such as Spark or Flink.
- Familiarity with orchestration frameworks.
- Experience with distributed OLAP datastores such as Druid or ClickHouse.
- Hands-on experience with ELK stack (Elasticsearch, Logstash, Kibana) for log aggregation, analysis.

Preferred Qualifications:
- Experience with big data ecosystems such as Hadoop, Hive, Spark, or similar.
- Solid understanding of software engineering best practices in the context of data services and large-scale systems.
- Enjoys solving complex data problems and creating scalable infrastructure to support analytical products.
- Passion for enabling advanced analytics and machine learning through high-quality, well-structured data.