Senior Data Engineer
Job Summary
Job Description
As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines, building and maintaining data storage systems, and collaborating with data analysts and scientists. The role involves following data governance policies, participating in code reviews, and ensuring solutions are production-ready. You'll work with Big Data Solution Architects to optimize data ingestion and provide technical support. Effective communication about project status and blockers is essential. The candidate will need to have experience with SQL, Python, and cloud technologies like Azure, AWS, or GCP. Proven skills in real-time processing, batch processing, and data orchestration are needed.
Must have:
- 5+ years of experience with big data technologies.
- Proficiency in SQL and Python.
- Experience in cloud technologies like Azure, AWS, GCP.
- Experience with Docker, Helm, Kubernetes.
- Skills in real-time Processing: Kafka, Apache Flink, Apache Spark.
- Experience with Batch Processing: Apache Spark (PySpark).
- Experience with Data Orchestration: Azure Data Factory, Apache Airflow.
- Experience with Data Storage: Parquet, Delta Lake, Iceburg.
- Experience with DevOps & CI/CD: Git, Docker, Terraform or Ansibel.
- Skills in querying and manipulating data from various databases.
- Experience in writing unit and integration tests.
- Excellent communication, problem-solving, and leadership skills.
Good to have:
- Experience with data visualization tools (e.g., SSRS, Power BI).
- Knowledge of machine learning algorithms and their applications.
- Familiarity with data privacy regulations (e.g., GDPR, CCPA).
- Experience with Great Expectations
Perks:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
21 skills required
Job Details
N-iX is looking for a talented Senior Data Engineer to join our team!
Our client is one of Europe's fastest-growing mobile companies with almost 5 million active customers, 1,000 employees worldwide, and operations in five countries. The client is using the mobile virtual network operator business model.
Responsibilities:
- Design and implement data pipelines to collect, clean, and transform data from various sources
- Build and maintain data storage and processing systems, including databases, data warehouses, and data lakes
- Follow data governance policies and procedures
- Collaborate with data analysts, data scientists, and other stakeholders to understand and meet their data needs
- Participate in code reviews, performance tuning, and best practice discussions within the team, and brainstorming sessions
- Work with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines
- Ensure solutions are production-ready in terms of operational, security, and compliance standards
- Participate in daily project and agile meetings, providing technical support for issue resolution
- Communicate clearly and concisely with the business about item status and blockers
- Maintain comprehensive knowledge of the client's data landscape
Requirements:
- 5+ years of design & development experience with big data technologies
- Proficiency in SQL and Python
- Experience in cloud technologies like Azure, AWS, GCP
- Experience with Docker, Helm, Kubernetes
- Proven skills in real-time Processing: Kafka, Apache Flink, Apache Spark Structured Streaming
- Experience with Batch Processing: Apache Spark (PySpark)
- Data Orchestration: Azure Data Factory, Apache Airflow,
- Data Storage: Parquet, Delta Lake, Iceburg
- DevOps & CI/CD: Git, Docker, Terraform or Ansibel (basic infra provisioning)
- Strong skills in querying and manipulating data from various databases (relational and big data)
- Experience in writing effective and maintainable unit and integration tests for ingestion pipelines
- Excellent communication, problem-solving, and leadership skills
- Experience working on high-traffic and large-scale software products
- Strong intermediate level of English
Nice to have:
- Experience with data visualization tools (e.g., SSRS, Power BI)
- Knowledge of machine learning algorithms and their applications in big data
- Familiarity with data privacy regulations (e.g., GDPR, CCPA)
- Experience with Great Expectations
We offer*:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
*not applicable for freelancers
Similar Jobs
Lytx, Inc
San Diego, California, United States (On-Site)
• 3 Weeks ago
Apple
Austin, Texas, United States (On-Site)
• 2 Weeks ago
Get notifed when new similar jobs are uploaded
Similar Skill Jobs
whoop
Boston, Massachusetts, United States (On-Site)
• 3 Months ago
NCR Voyix
Chennai, Tamil Nadu, India (On-Site)
• 4 Weeks ago
Palo Alto Networks
Santa Clara, California, United States (On-Site)
• 2 Months ago
bytedance
San Jose, California, United States (On-Site)
• 8 Months ago
Get notifed when new similar jobs are uploaded
Jobs in Ukraine
Get notifed when new similar jobs are uploaded
Data Analysis Jobs
bytedance
San Jose, California, United States (On-Site)
• 8 Months ago
Get notifed when new similar jobs are uploaded
