This Senior PySpark Data Engineer role involves collaborating with business SMEs to implement data validation rules, understanding business requirements and data flow, and working with PySpark-based data platforms like Databricks and Cloudera.  The position requires proficiency in PySpark, Python, and SQL, along with knowledge of Azure data/delta lake, Azure DevOps, CI/CD, and Git.  Responsibilities include understanding the data platform lifecycle, contributing to design documents, unit test plans, and code reviews, and working in an Agile environment.  The project is located in the Middle East and involves working with a team of front-end and back-end developers, data analysts, data scientists, architects, and project managers.

<p><span style="color: rgba(0, 0, 0, 0.6);">Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. </span></p>

Luxoft

Senior QA Automation Python Engineer

Senior ML Engineer

Senior Java Developer

Senior React Developer

Technical Business Analyst

Senior Angular Developer

Solution Architect

DevOps Engineer with Azure

<div class="job__grid__about-job">
                <div class="job__grid__about-job__description">
                    <h5 class="job__grid__about-job__description__title">
                        Project description
                    </h5>
                    <p class="body-l-regular job__grid__about-job__description__text">
                         We are hiring for projects in the Middle East. The team comprises front-end and back-end developers, data analysts and data scientists, architects, analysts, and project managers. You will work with modern technologies and be an integral part of continuing to evolve our high-performing platforms. If you are serious about being a part of this fast-paced and exciting journey, this project should be top of the list!<br>This role will be primarily responsible for closely collaborating with a business SME to implement data validation rules.
                    </p>
                </div>
                <div class="job__grid__about-job__responsabilities">
                    <h5>Responsibilities</h5>
                                        <div class="job__grid__about-job__responsabilities__list">
                                                   <div class="job__grid__about-job__responsabilities__list__item">
                            
                            <p class="body-l-regular">
                                Collaborating with a business SME to implement data validation rules.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__responsabilities__list__item">
                            
                            <p class="body-l-regular">
                                Understand business requirements and actively provide input from a data perspective.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__responsabilities__list__item">
                            
                            <p class="body-l-regular">
                                Understand the underlying data and flow of data.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__responsabilities__list__item">
                            
                            <p class="body-l-regular">
                                Understand overall platform architecture and capabilities and how to load data effectively.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__responsabilities__list__item">
                            
                            <p class="body-l-regular">
                                Recognize and adapt to the changes in processes.
                            </p>
                        </div>
                                             </div>
                </div>
                <div class="job__grid__about-job__skills">
                    <h5>Skills</h5>
                    <p class="body-xl-semibold job__grid__about-job__skills--subtitle">
                        Must have
                    </p>
                                        <div class="job__grid__about-job__skills__list">
                                                   <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                7 years + of relevant experience.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Proficient hands-on experience with PySpark-based data platforms:
                            </p>
                        </div>
                                                  <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Databricks, Cloudera, Apache Spark,  etc. (3y+).
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Proficiency in Python and PySpark (5y+).
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Advanced knowledge in SQL (5y+).
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Should be able to analyse and understand complex data.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Knowledge of Azure data/delta lake, Azure DevOps, CI/CD, and Git is essential.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Should clearly understand the data platform lifecycle and contribute to preparing Design documents, Unit Test plans, and Code review reports.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Knowledge of data Lake house and business intelligence architecture.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Experience working in an Agile environment (Scrum, Agile) is a plus.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Interpersonal/communication skills (both oral/written) with the ability to communicate at various levels with clarity &amp; precision.
                            </p>
                        </div>
                                                    <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Bachelor's degree in IT/Computer Science or related field
                            </p>
                        </div>
                                             </div>
                    <p class="body-xl-semibold job__grid__about-job__skills--subtitle">
                        Nice to have
                    </p>
                                        <div class="job__grid__about-job__skills__list">
                                                 <div class="job__grid__about-job__skills__list__item">
                            
                            <p class="body-l-regular">
                                Knowledge of the financial markets, portfolio theory, and risk management is a plus
                            </p>
                        </div>
                                             </div>
                </div>

                <div class="job__grid__about-job__other">
                    <h5>Other</h5>
                    <div class="job__grid__about-job__other__items">
                        <div class="job__grid__about-job__other__items__item">
                            
                            <div class="job__grid__about-job__other__items__item__text">
                                <p class="caption-xs-bold">Languages</p>
                                <p class="body-l-regular">
                                    English: C1 Advanced
                                </p>
                            </div>
                        </div>
                        <div class="job__grid__about-job__other__items__item">
                            
                            <div class="job__grid__about-job__other__items__item__text">
                                <p class="caption-xs-bold">Seniority</p>
                                <p class="body-l-regular">
                                    Senior
                                </p>
                            </div>
                        </div>
                    </div>
                </div>
            </div>