Data Engineer

Description

• Design, develop, and maintain Analytics/dashboards/Automation and reports.
• Work closely with stakeholders to understand their needs and translate them into business/DW solutions.
• Ensure the performance optimisation, quality, and responsiveness of the dashboards.
• Collaborate with the team to define, design, and ship new features.
• Identify and correct bottlenecks and fix bugs.
• Help maintain code quality, organisation, and automatisation.

• Strong understanding of Datawarehouse Concepts. 
• Experience translating theoretical concepts into engineered features, with a demonstrated ability to create features capturing important events or transitions within the data.
• Expertise in crafting custom features that highlight unique patterns specific to the dataset or problem, enhancing model predictive power. Ability to combine and synthesise information from multiple data sources to develop more informative features.
• Advanced knowledge in Apache Spark (PySpark, SparkSQL, SparkR) and distributed computing, demonstrated through efficient processing and analysis of large-scale datasets. Proficiency in Python, R, and SQL, with a proven track record of writing optimized and efficient Spark code for data processing and model training.
• Hands-on experience with cloud-based machine learning platforms such as AWS SageMaker and Databricks, showcasing scalable model development and deployment.
• Demonstrated capability to develop and implement custom statistical algorithms.
• Proficiency in statistical methods for identifying patterns and trends in large datasets, essential for predictive maintenance.
• Experience optimising code for performance in high-throughput, low-latency environments. Experience deploying models into production, with expertise in monitoring their performance and integrating them with CI/CD pipelines using AWS, Docker, or Kubernetes.
• Familiarity with end-to-end analytical architectures, including data lakes, data warehouses, and real-time processing systems.
• Experience creating insightful dashboards and reports using tools such as Power BI, Tableau, Incorta or custom visualisation frameworks to effectively communicate model results to stakeholders.
• 5+ years of experience in data science and explored Gen AI/DL/AIML model using ML Flows

• Bachelor’s or Master’s degree/ Diploma in Engineering, Statistics, Mathematics or Computer Science
• 5+ years of experience as a Data Engineer/Data Science
• Strong problem-solving skills
• Proven ability to work independently and accurately
• Strong Expertise in Python, Spark, R, SQL or any other programming language
• Strong hands-on expertise with at least one analytics tools like Snowflake, Databricks or Incorta.