Data Engineer (AWS stack)

1 Day ago • 4 Years +

Job Summary

Job Description

PayPay India is seeking a Platform Data Engineer to develop and deploy data pipelines and manage data at scale on mission-critical systems. Responsibilities include designing and maintaining ETL/ELT data pipelines, working with various data types, utilizing engineering best practices, advising on cloud resource utilization, collaborating with teams, and documenting integration processes. The role requires ensuring compliance with data governance policies and industry standards.
Must have:
  • 3+ years of AWS experience
  • 4+ years experience as a Data Engineer
  • Strong problem-solving skills for data issues
  • Hands-on experience with Infra as Code (terraform)
  • Excellent communication and collaboration skills
  • Ability to manage multiple tasks simultaneously
Good to have:
  • Experience with Cloud services (EC2, S3, EMR, etc.)
  • Design and implement data integration workflows
  • Ensure data quality and integrity across systems
  • Validate and cleanse data
  • Optimize performance of Elasticsearch/Opensearch clusters
  • Experience in a primary language (e.g., Scala, Python, Java) and SQL

Job Details

 

 

About PayPay India

PayPay, a fintech company providing a service enjoyed by over 66 million users (as of Oct 2024) merely 6 years since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society.
 
 
 
 

Why India ?

To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated.

OUR VISION IS UNLIMITED

We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined.

Job Description

PayPay India is looking for Platform Data Engineer who can develop and deploy data pipelines and manage data at scale on mission critical systems.

Main Responsibilities :

  • Design, build and maintain ETL/ELT data pipeline to process and ingest data from multiple sources.
  • Work with structured and semi/un-structured data.
  • Utilizing your skills in engineering best practices to solve complex data problems.
  • Advising on specific technologies and methodologies for utilising cloud resources to efficiently ingest and process data quickly.
  • Collaborating with teams to ensure that prospective data architecture plans maximize the value of data across the organization.
  • Document all data integration processes , workflows and technical & system specifications.
  • Ensure compliance with data governance policies , industry standards and regulatory requirements.

Tech Stack :

We select the best combination of tech at times.
| Python, Shell, Go, SQL, PL/SQL

| Aurora, MySQL, TiDB, Redis, DynamoDB

| AWS

| GitHub, Terraform, Jenkins, Ansible, Flyway

| Victoria Metrics, Grafana, Prometheus, Newrelic

| Elasticsearch , Opensearch , Athena , Glue , EMR

 

Qualifications

  • More than 3 years of AWS experience
  • 4+ years experience as a Data Engineer or in a similar role
  • Strong problem-solving skills and ability to troubleshoot complex data issues
  • Hands-on experience with Infra as Code (IaC - terraform)
  • Excellent communication and collaboration skills
  • Ability to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously

         

Preferred Qualifications

  • Experience designing and building solutions utilising various Cloud services such as EC2 , S3 , EMR , Kinesis , Glue , Athena , Lambda , Redshift , etc.,
  •  Design and implement data integration workflows and Ensuring data quality and integrity across systems.
  •  Validate and cleanse structured and semi/un-structured data to maintain high data quality.
  •  Monitor and optimize the performance of Elasticsearch / Opensearch clusters and Kafka clusters
  • . Experience in at least one primary language (e.g. Scala, Python, Java) and SQL (any variant).
  •  Stay update on cloud security best practices and apply them to the organization's environments.

 

Nice to Have :

  • Design and implement Elastic search / Open search clusters to meet the organization's search and analytics requirements.
  • Experience in operating and managing search and analytics engines like Elastic search , Open search , Logstash and Vector.
  • Design, implement and manage Kafka-based data pipelines and messaging solutions to support critical business operations and enable real-time data processing.
  • Experience in distributed event streaming platform and data processing framework.
  • Familiarity with Jenkins, Ansible, Docker, Kubernetes.
  • Experience in operating distributed systems.

 

Remarks

*Please note that you cannot apply for PayPay (Japan-based jobs) or other positions in parallel or in duplicate.

PayPay 5 senses


Working Conditions 

Employment Status

  • Full Time

Office Location

  • Gurugram (Wework)

    *The development center requires you to work in the Gurugram office to establish the strong core team.
       

Similar Jobs

ION - Principal Technical Consultant - Endur

ION

London, England, United Kingdom (On-Site)
6 Months ago
Zscaler - Staff Software Engineer - Golang

Zscaler

San Jose, California, United States (On-Site)
9 Hours ago
Gunzilla - Blockchain Technical Support Engineer

Gunzilla

Frankfurt, Hessen, Germany (On-Site)
1 Month ago
Aristocrat Gaming - Senior Software Developer

Aristocrat Gaming

London, England, United Kingdom (Hybrid)
4 Months ago
The Walt Disney Company - Lead Software Engineer

The Walt Disney Company

Seattle, Washington, United States (On-Site)
1 Month ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Playrix - Principal Golang Engineer (Cross-Game Server)

Playrix

Ireland (Remote)
2 Weeks ago
Glean - Technical Support Engineer ( Japan)

Glean

Japan (Remote)
3 Months ago
Electronic Arts - Senior AI Data Scientist

Electronic Arts

Kirkland, Washington, United States (On-Site)
1 Day ago
White Hat Gaming  - SRE/DevOps Engineer

White Hat Gaming

(Remote)
2 Months ago
NVIDIA - Senior Software Engineer - Distributed Inference

NVIDIA

California, United States (Remote)
1 Month ago
Flexra Software - Security Operations Engineer

Flexra Software

Bengaluru, Karnataka, India (Hybrid)
1 Day ago
Plarium - Data Engineer

Plarium

Herzliya, Tel Aviv District, Israel (Hybrid)
9 Months ago
Bad Robot Games - Online Engineer

Bad Robot Games

California, United States (Remote)
2 Weeks ago
Eventbrite - Site Reliability Engineer II

Eventbrite

(Remote)
1 Day ago
Cyara - Sr. Software Engineer

Cyara

Hyderabad, Telangana, India (Hybrid)
3 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Gurugram, India

Take-Two Interactive - Manager, People Operations

Take-Two Interactive

Bengaluru, Karnataka, India (Hybrid)
1 Day ago
PwC - IN_Senior Associate _Data  Migration_Enterprise_Apps_SAP_Kolkata

PwC

Kolkata, West Bengal, India (On-Site)
7 Months ago
BOT VFX - Associate Head of Creative Operations

BOT VFX

Chennai, Tamil Nadu, India (On-Site)
3 Weeks ago
Hitachi - Share Point Developer

Hitachi

Pune, Maharashtra, India (Remote)
6 Months ago
Arcadia - Senior Engineering Manager - Python (RUDE squad)

Arcadia

Chennai, Tamil Nadu, India (Hybrid)
8 Hours ago
PhonePe - Gujarati Linguist

PhonePe

Bengaluru, Karnataka, India (On-Site)
8 Hours ago
Google - Head of Music Partnerships, Artist and Label Relations

Google

Mumbai, Maharashtra, India (On-Site)
2 Weeks ago
Google - Strategy and Operations Lead, gTech Ads Solutions

Google

Hyderabad, Telangana, India (On-Site)
2 Days ago
Google - Vendor Operations Manager, YouTube Global Vendor Operations

Google

Hyderabad, Telangana, India (On-Site)
2 Weeks ago
Synechron - Senior Developer (Python, Java, and IBM DataStage)

Synechron

Pune, Maharashtra, India (On-Site)
1 Day ago

Get notifed when new similar jobs are uploaded

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

Gurugram, India (On-Site)

Gurugram, India (On-Site)

Gurugram, India (On-Site)

Gurugram, India (On-Site)

Gurugram, India (On-Site)

Gurugram, India (On-Site)

Gurugram, India (On-Site)

Gurugram, India (On-Site)

View All Jobs

Get notified when new jobs are added by pay2dc

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug