Jobs

Scholarships

Courses

Companies

Placements

Community

Home >

Jobs >

Data Engineer - Python

Karnataka, India (On-site)

Data Engineer - Python

2 Weeks ago • 4-6 Years • Data Analyst

About the job

14 skills required for this role

Test your skills to join the top 1% applicants for this job

shell

jenkins

github-actions

cassandra

ci-cd

github

kubernetes

openshift

nosql

aws

python

docker

sql

git

Job Description

Develop end-to-end solutions for a major investment banking product, scaling the tech stack using cloud services within the Payment Settlement stream. Responsibilities include writing efficient, scalable Python code for data management (extraction, mining, analysis, validation, storage), migrating legacy shell scripts to optimized Python, testing and debugging, managing AWS/GCP pipelines with Python, coordinating with internal teams, and understanding existing script logic for Python migration. The role requires deep Python expertise, SQL database proficiency, experience with RDDs, various SQL/NoSQL databases (Postgres/Cassandra/Elasticsearch/MongoDB), data quality framework building, and microservice deployment using Docker/Kubernetes/OpenShift. CI/CD pipeline understanding (Jenkins/Git/GitHub/GitHub Actions) is crucial.

Must have:

4-6 years Python development experience
Deep understanding of Python and its ecosystem
Proficient in SQL database programming with Python
Experience with Resilient Distributed Datasets (RDDs)
Experience with various SQL/NoSQL databases
Data Quality Framework building experience
Microservice deployment in Docker/Kubernetes/OpenShift
CI/CD pipeline understanding (Jenkins/Git/GitHub/GitHub Actions)

Good to have:

Data Engineering or ML certification
AI/ML pipeline building experience
Investment banking project experience

Project description

Provide End to End Development solution to one of the largest investment banking products by scaling up the latest technology stack by adopting the cloud-based services under Payment Settlement stream.

Responsibilities

Write effective, scalable code using Python.

In-depth working knowledge of data management activities like data extraction, data mining, data analysis, data validation, and data storage.

Migrate legacy shell scripts to Python using optimized code standards to improve performance and error handling.

Test and debug the programs.

Managing the pipelines in AWS/GCP using Python code.

Coordinate with internal teams to understand user requirements and provide technical solutions.

Understand the current logic of the scripts and migrate it to Python.

Skills

Must have

4-6 Years of Experience working on Python Development projects

Deep understanding of Python's language, ecosystem, features, and experience in designing and developing data pipelines

The ability to program SQL databases with Python

Experience in interfacing with Resilient Distributed Datasets (RDDs) in Python

Solid experience in building queries in various SQL/NoSQL databases (Postgres/Cassandra/Elastic Search/MangoDB)

Experience in building a Data Quality Framework on various types of Datasets

Experience in deploying the MicroService in Docker/Kubernetes/OpenShift

Understanding of CI/CD Pipelines through Jenkins/Git/Github/Github Actions

Nice to have

Certification in Data Engineering or ML is an added advantage

Knowledge or working experience in building AI or ML pipeline

Experience in investment banking-related projects

Other

Languages

English: C1 Advanced

Seniority

Senior

View Full Job Description

Upload your resume, increase your shortlisting chances by 80%

About The Company

Luxoft

1148 Active Jobs

Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 21 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications.

DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.