Senior Data Engineer, AI Applications

Let's start with the role:

At Kaizen, we aim to leverage the capabilities offered by modern AI algorithms to continuously improve our product and business. Kaizen’s AI department consists of product, platform and research teams and lies at the core of that mission. We develop AI products for various applications, ranging from detection of fraudulent activity to personalized gaming recommendations, and from AI-powered co-pilots to CRM optimizations. 

We build our teams to be self-driven, autonomous and cross-functional. Within these teams, you will have the opportunity to work with Data Scientists, Machine Learning Engineers, Data Engineers, Product Analysts, Product Owners and Agile Delivery Leads.

As a Senior Data Engineer, you will focus on delivering robust and scalable data pipelines which transform raw data to features that can be used in Machine Learning models. 

As a Senior Data Engineer your will:

• Design, develop and maintain scalable data processing pipelines in a distributed, and data-intensive production environment;
• Design, develop and maintain microservices in Python that serve these features on production;
• Design, develop and maintain internal tools that enable CI/CD pipelines, experiment tracking and data versioning;
• Guarantee data integrity by employing data quality processes.

What you’ll bring:

Must have:

• Significant experience in software development using Python on a large scale, high-performance production environment;
• Significant hands-on experience with Spark and PySpark;
• Significant experience with data modeling and handling structured and unstructured data;
• Significant experience with Apache Kafka;

• Experience with cloud environments;
• Expert skills in Distributed Software System design;
• Excellent problem solving and communication skills.

Nice to have:

• Experience with the Databricks Platform;
• Experience with streaming data and real-time data processing frameworks;
• Experience with NoSQL DBs, such as Redis and Neo4j;
• Good understanding of ML algorithms and the ML-project lifecycle.