Outscal Logooutscal logo
Company logo
Luxoft
Karnataka, India (On-site)

Data Engineer - Python

9 Months ago • 4-6 Years

Job Description

Develop end-to-end solutions for a major investment banking product, scaling the tech stack using cloud services within the Payment Settlement stream. Responsibilities include writing efficient, scalable Python code for data management (extraction, mining, analysis, validation, storage), migrating legacy shell scripts to optimized Python, testing and debugging, managing AWS/GCP pipelines with Python, coordinating with internal teams, and understanding existing script logic for Python migration. The role requires deep Python expertise, SQL database proficiency, experience with RDDs, various SQL/NoSQL databases (Postgres/Cassandra/Elasticsearch/MongoDB), data quality framework building, and microservice deployment using Docker/Kubernetes/OpenShift. CI/CD pipeline understanding (Jenkins/Git/GitHub/GitHub Actions) is crucial.
Good To Have:
  • Data Engineering or ML certification
  • AI/ML pipeline building experience
  • Investment banking project experience
Must Have:
  • 4-6 years Python development experience
  • Deep understanding of Python and its ecosystem
  • Proficient in SQL database programming with Python
  • Experience with Resilient Distributed Datasets (RDDs)
  • Experience with various SQL/NoSQL databases
  • Data Quality Framework building experience
  • Microservice deployment in Docker/Kubernetes/OpenShift
  • CI/CD pipeline understanding (Jenkins/Git/GitHub/GitHub Actions)

Add these skills to join the top 1% applicants for this job

shell
jenkins
github-actions
cassandra
ci-cd
github
kubernetes
openshift
nosql
aws
python
docker
sql
git
data-analytics

Project description

Provide End to End Development solution to one of the largest investment banking products by scaling up the latest technology stack by adopting the cloud-based services under Payment Settlement stream.

Responsibilities

Write effective, scalable code using Python.

In-depth working knowledge of data management activities like data extraction, data mining, data analysis, data validation, and data storage.

Migrate legacy shell scripts to Python using optimized code standards to improve performance and error handling.

Test and debug the programs.

Managing the pipelines in AWS/GCP using Python code.

Coordinate with internal teams to understand user requirements and provide technical solutions.

Understand the current logic of the scripts and migrate it to Python.

Skills

Must have

4-6 Years of Experience working on Python Development projects

Deep understanding of Python's language, ecosystem, features, and experience in designing and developing data pipelines

The ability to program SQL databases with Python

Experience in interfacing with Resilient Distributed Datasets (RDDs) in Python

Solid experience in building queries in various SQL/NoSQL databases (Postgres/Cassandra/Elastic Search/MangoDB)

Experience in building a Data Quality Framework on various types of Datasets

Experience in deploying the MicroService in Docker/Kubernetes/OpenShift

Understanding of CI/CD Pipelines through Jenkins/Git/Github/Github Actions

Nice to have

Certification in Data Engineering or ML is an added advantage

Knowledge or working experience in building AI or ML pipeline

Experience in investment banking-related projects

Other

Languages

English: C1 Advanced

Seniority

Senior

Set alerts for new jobs by Luxoft
Set alerts for new jobs in India

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug
Company
Contact Us
hello@outscal.com
Made in INDIA 💛💙