Senior Data Engineer

N-iX is looking for a Senior Data Engineer with strong expertise in PySpark to join our team in Kyiv. In this role, you will design, develop, and optimize large-scale data pipelines, ensuring efficient data processing and integration. You will also collaborate closely with business stakeholders to gather and refine data requirements, ensuring solutions align with business needs. Experience with Palantir Foundry is a huge plus.

Responsibilities:

• Design and implement scalable ETL/ELT pipelines using PySpark.
• Work closely with product owners, and different stakeholders to gather, analyze, and translate data requirements into technical solutions.
• Optimize data processing workflows for performance and reliability.
• Ensure data integrity, governance, and security best practices.
• Develop and maintain technical documentation for data pipelines and transformations.
• Troubleshoot and optimize Spark jobs to improve efficiency.
• Work with cloud-based or on-premises big data technologies.

Requirements:

• 5+ years of experience in data engineering, with a focus on PySpark.
• Strong knowledge of Apache Spark, distributed computing, and performance tuning.
• Experience gathering and refining business and technical data requirements.
• Hands-on experience with SQL, data modeling, and ETL processes.
• Proficiency in working with big data technologies (Hadoop, Databricks, etc.).
• Experience with cloud platforms (AWS, Azure, GCP) is a plus.
• Knowledge of CI/CD, version control (Git), and DevOps practices.
• Excellent problem-solving and communication skills.

Nice to have:

• Experience with Palantir Foundry.
• Familiarity with streaming technologies (Kafka).
• Exposure to containerization and orchestration (Docker, Kubernetes, Airflow).