Data Engineer (Python, PySpark, AWS)

Job Summary:

We are looking for an experienced Data Engineer with strong hands-on expertise in Python, PySpark, and AWS Cloud services. The ideal candidate will play a key role in building scalable, efficient, and reliable data pipelines and supporting data infrastructure for analytics and reporting.

Key Responsibilities:

• Design, build, and maintain scalable data pipelines using PySpark and Python
• Develop and optimize ETL workflows to process structured and semi-structured data
• Work with large datasets stored on AWS S3, and process them using Glue, EMR, or Lambda
• Collaborate with Data Scientists, Analysts, and Product teams to define data needs and deliver quality solutions
• Ensure data quality, integrity, and consistency across systems
• Manage orchestration using AWS Step Functions, Glue Workflows, or Airflow
• Monitor and optimize performance of pipelines, storage, and compute resources
• Participate in data modeling, schema design, and documentation

Required Skills:

• 4+ years of experience as a Data Engineer or similar role
• Strong programming skills in Python with experience in writing clean, scalable code
• Hands-on experience with PySpark and distributed data processing
• Proficient in AWS services: S3, Glue, EMR, Lambda, Athena, Redshift
• Good understanding of SQL and working with relational databases
• Familiarity with DevOps tools (Git, CI/CD) and job scheduling/orchestration tools
• Strong problem-solving skills and attention to detail

Educational Qualification:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field