SENIOR DATA ENGINEER
Job Summary
Job Description
We are looking for an experienced Data Engineer to help us build and support data processing pipelines on the Spark on Kubernetes stack (Scala and SQL API), Airflow, Clickhouse and Python. You will be responsible for optimizing infrastructure, helping analysts build complex data processing pipelines in Airflow, and organizing processes for systematic data collection from different sources. You will also be responsible for developing an infrastructure for solving product analytics problems based on ClickHouse.
Must have:
- Experience as a Data Engineer
- Build ETL on Apache Spark
- Implement orchestration on Apache Airflow
- SQL and Python
- S3
- Databases (ClickHouse, PostgreSQL, MySQL)
- Query optimization
- Fluent Russian
Perks:
- Relocation to Serbia, Armenia, or another destination
- Relocation program and assistance with paperwork
- Competitive salary, reviewed every 6 months
- Flexible work setup
- Flexible benefits
- Full pay during sick leaves and personal leaves
- 20 working days of vacation annually
11 skills required
Job Details
Our team values an analytical approach to decision making. We accumulate a huge amount of data every day – game events, marketing metrics and much more, which, when carefully analyzed, helps us get valuable insights and improve our games. High quality and speed of processing of this data play a key role for us. And we are looking for an experienced Data Engineer who will help us with this and will be able to take our work with data to the next level.
What we will trust you with
- Building and supporting data processing pipelines on the Spark on Kubernetes stack (Scala and SQL API), Airflow, Clickhouse and Python.
- Optimization of infrastructure (optimal selection of memory size, cores, disks, instance types) for data processing tasks.
- Helping analysts build complex data processing pipelines in Airflow.
- Organization of processes for systematic data collection from different sources using different protocols.
- Development of an infrastructure for solving product analytics problems based on ClickHouse.
Our stack:
- Orchestration - AirFlow v2.5.3
- ETL - Spark v3.2.0, AWS Glue
- Resource Management - Kubernetes
- Storage - AWS S3
- DB - ClickHouse, PostgreSQL, MySQL
- Languages - Scala, Python, SQL
- Data governance - DataHub
- DQ - Deequ
- CI CD - GitLab CI
- Secret storage - Vault
- BI - Tableau
What we expect from you
- Experience as a Data Engineer or similar position for at least three years.
- Ability to build ETL on Apache Spark.
- Understanding of how to implement orchestration on Apache Airflow.
- Knowledge of SQL and the ability to understand Python code.
- Experience working with S3.
- Experience in working with databases (ClickHouse, PostgreSQL, MySQL) and query optimization.
- Fluent Russian.
WHAT WE OFFER
- An opportunity to contribute significantly to a globally successful project within a stable international company.
- Relocation to Serbia, or Armenia, or another destination upon our mutual agreement.
- Relocation program and assistance with paperwork for residence in countries where our offices are located.
- Competitive salary, reviewed every 6 months based on market trends and your performance.
- Flexible work setup: whether you prefer coming to the office, working from home, or a hybrid of the two, the choice is yours.
- Flexible benefits: an annual $1000 to spend on perks, including health insurance, sports, therapy, education and language classes.
- Full pay during sick leaves and personal leaves, in addition to 20 working days of vacation annually.
Similar Jobs
The Walt Disney Company
San Francisco, California, United States (On-Site)
• 3 Months ago
Canva
Auckland, Auckland, New Zealand (Remote)
• 3 Months ago
The Walt Disney Company
Burbank, California, United States (On-Site)
• 3 Months ago
PENN Interactive
Philadelphia, Pennsylvania, United States (Hybrid)
• 3 Months ago
Get notifed when new similar jobs are uploaded
Similar Skill Jobs
Nielsen Holdings
Gurugram, Haryana, India (Hybrid)
• 4 Months ago
Warner Bros Games
Bengaluru, Karnataka, India (Hybrid)
• 4 Months ago
Nielsen Holdings
Bengaluru, Karnataka, India (Hybrid)
• 4 Months ago
Canva
Melbourne, Victoria, Australia (Remote)
• 3 Months ago
The Walt Disney Company
Seattle, Washington, United States (On-Site)
• 3 Months ago
PwC
Bengaluru, Karnataka, India (On-Site)
• 4 Months ago
The Walt Disney Company
Seattle, Washington, United States (On-Site)
• 3 Months ago
Get notifed when new similar jobs are uploaded
Jobs in Belgrade, Serbia
Wargaming
Belgrade, Serbia (On-Site)
• 3 Months ago
Get notifed when new similar jobs are uploaded
Data Analyst Jobs
Life church
Edmond, Oklahoma, United States (On-Site)
• 4 Months ago
Match Group
Dallas, Texas, United States (Hybrid)
• 4 Months ago
Get notifed when new similar jobs are uploaded
