SRE- Hadoop

3 Months ago • 4-6 Years • Administrative

Job Summary

Job Description

As an SRE-Hadoop at PhonePe, you'll be responsible for the implementation and administration of Hadoop infrastructure, including HDFS, HBase, Hive, Pig, Airflow, YARN, Ranger, Kafka, and Druid. You will handle capacity planning, performance tuning, troubleshooting, and collaborating with various teams to ensure high data quality and availability. Responsibilities include developing tools for better supportability, applying security updates, and optimizing the Hadoop ecosystem's performance. Strong Linux, networking, and scripting skills are essential, along with experience in public cloud environments (Azure/AWS/GCP).
Must have:
  • 4+ years Hadoop administration experience
  • Strong Linux/Unix and networking skills
  • Public cloud experience (Azure/AWS/GCP)
  • Experience with Hadoop stack (HDFS, HBase, Hive, etc.)
  • Scripting (Perl, Golang, Python)
  • Troubleshooting and performance tuning
  • Collaboration with cross-functional teams
Good to have:
  • Airflow DAG development and maintenance
  • ELK stack administration
  • Kerberos and LDAP administration
  • Configuration management tools (Puppet, Salt, Ansible)
  • Experience with monitoring stacks (OpenTSDB, Prometheus, etc.)
Perks:
  • Medical Insurance
  • Critical Illness Insurance
  • Accidental Insurance
  • Life Insurance
  • Employee Assistance Program
  • Maternity/Paternity Benefits
  • Adoption Assistance
  • Relocation Benefits
  • Higher Education Assistance
  • Car Lease

Job Details

About PhonePe Group: 

PhonePe is India’s leading digital payments company with 50 crore (500 Million) registered users and 3.7 crore (37 Million) merchants covering over 99% of the postal codes across India. On the back of its leadership in digital payments, PhonePe has expanded into financial services (Insurance, Mutual Funds, Stock Broking, and Lending) as well as adjacent tech-enabled businesses such as Pincode for hyperlocal shopping and Indus App Store which is India's first localized App Store. The PhonePe Group is a portfolio of businesses aligned with the company's vision to offer every Indian an equal opportunity to accelerate their progress by unlocking the flow of money and access to services.

Culture

At PhonePe, we take extra care to make sure you give your best at work, Everyday! And creating the right  environment for you is just one of the things we do. We empower people and trust them to do the right  thing. Here, you own your work from start to finish, right from day one. Being enthusiastic about tech is a  big part of being at PhonePe. If you like building technology that impacts millions, ideating with some of  the best minds in the country and executing on your dreams with purpose and speed, join us!

JOB DESCRIPTION

Minimum of 1 year of experience in Linux/Unix Administration.

Minimum of 2 years of hands on experience with managing infra on public cloud i.e Azure/AWS/GCP

 Over 4+ years of experience in Hadoop administration.

Strong understanding of networking, open-source technologies, and tools.

Familiar with best practices and IT operations for maintaining always-up, always-available services.

Experience and participation during on-call rotation.Excellent communication skills.

Solid expertise in Linux networking, including IP, iptables, and IPsec.Proficient in scripting and coding with languages such as Perl, Golang, or Python.

Strong Knowledge of databases like Mysql,Nosql,Sql serverHand on experience with setting up , configuring and Managing Nginx as reverse proxy and load balancing in high traffic environments.

Hands-on experience with both private and public cloud environments.

Strong troubleshooting skills and operational expertise in areas such as system capacity, bottlenecks, memory, CPU, OS, storage, and networking.

Practical experience with the Hadoop stack, including Hdfs,HBase,Hive, Pig, Airflow, YARN, HDFS, Ranger, Kafka, and Druid.

Good to have experience with Design,develop and maintain Airflow DAGs and tasks to automate BAU processes,ensuring they are robust,scalable and efficient.

Good to have experience with ELK stack administration.

Experience in administering Kerberos and LDAP.

Familiarity with open-source configuration management and deployment tools like Puppet, Salt, or Ansible.Responsible for the implementation and ongoing administration of Hadoop infrastructure.

Experience in capacity planning and performance tuning of Hadoop clusters.

Collaborate effectively with infrastructure, network, database, application, and business intelligence teams to ensure high data quality and availability.

Develop tools and services to enhance debuggability and supportability.Work closely with security teams to apply Hadoop updates, OS patches, and version upgrades as needed.

Troubleshoot complex production issues, identify root causes, and provide mitigation strategies.

Work closely with teams to optimize the overall performance of the PhonePe Hadoop ecosystem.

Experience with setting up & managing monitoring stack like OpenTsdb,Prometheus,ELK,Grafana,Loki

PhonePe Full Time Employee Benefits (Not applicable for Intern or Contract Roles)

  • Insurance Benefits - Medical Insurance, Critical Illness Insurance, Accidental Insurance, Life Insurance
  • Wellness Program - Employee Assistance Program, Onsite Medical Center, Emergency Support System
  • Parental Support - Maternity Benefit, Paternity Benefit Program, Adoption Assistance Program, Day-care Support Program
  • Mobility Benefits - Relocation benefits, Transfer Support Policy, Travel Policy
  • Retirement Benefits - Employee PF Contribution, Flexible PF Contribution, Gratuity, NPS, Leave Encashment 
  • Other Benefits - Higher Education Assistance, Car Lease, Salary Advance Policy

Working at PhonePe is a rewarding experience! Great people, a work environment that thrives on creativity, the opportunity to take on roles beyond a defined job description are just some of the reasons you should work with us. Read more about PhonePe on our blog.

Life at PhonePe

PhonePe in the news

Similar Jobs

CloudHire - Sr. Database Engineer

CloudHire

India (Remote)
3 Months ago
PwC - IN-Senior Associate_Big Data Engineer_Data & Analytics_Advisory_ PAN India

PwC

Gurugram, Haryana, India (On-Site)
4 Months ago
Appier - Senior Software Engineer, Data Backend(CrossX)

Appier

Taipei City, Taiwan (On-Site)
3 Months ago
Paypal - Senior Product Data Scientist

Paypal

San Jose, California, United States (Hybrid)
3 Months ago
PwC - Associate

PwC

Pune, Maharashtra, India (On-Site)
4 Months ago
Aristocrat Gaming - Project Manager

Aristocrat Gaming

Warsaw, Masovian Voivodeship, Poland (Hybrid)
1 Month ago
Corsair - Payroll Specialist

Corsair

Taoyuan City, Taiwan (On-Site)
1 Month ago
Keywords Studios (Player Support) - Technical Support Engineer II

Keywords Studios (Player Support)

Pune, Maharashtra, India (Hybrid)
1 Month ago
PhonePe - Site Reliability Engineer - Systems

PhonePe

Bengaluru, Karnataka, India (On-Site)
3 Months ago
Illumina - IT Engineer- Data Protection

Illumina

Bengaluru, Karnataka, India (On-Site)
4 Months ago

Get notifed when new similar jobs are uploaded

Similar Skill Jobs

Paypal - Sr. Staff Machine Learning Scientist

Paypal

San Jose, California, United States (On-Site)
4 Months ago
RoofStack - Data Architect

RoofStack

İstanbul, İstanbul, Türkiye (Remote)
1 Month ago
EXUSIA - Ab Initio Technical Lead PUN

EXUSIA

India (Remote)
4 Months ago
ByteDance - Backend Engineer - Applied Machine Learning Platform

ByteDance

Singapore (On-Site)
3 Months ago
Rackspace Technology - Presales Data Science Architect – AWS Cloud

Rackspace Technology

Mexico City, Mexico (On-Site)
3 Months ago
UXBERT Labs - Senior Data Engineer

UXBERT Labs

Riyadh, Riyadh Province, Saudi Arabia (Hybrid)
1 Month ago
BigID - Sr Solutions/Presales Engineer - West

BigID

Denver, Colorado, United States (Remote)
3 Months ago
Nielsen Holdings - SOFTWARE DEVELOPER - (Backend Developer, Java,Scala, Python, GO lang)

Nielsen Holdings

Bengaluru, Karnataka, India (Hybrid)
4 Months ago
The Walt Disney Company - Senior Data Engineer - Identity Data

The Walt Disney Company

New York, New York, United States (On-Site)
2 Months ago

Get notifed when new similar jobs are uploaded

Jobs in Bengaluru, Karnataka, India

Essvee Consultant - Interior Designer

Essvee Consultant

Gurugram, Haryana, India (On-Site)
4 Months ago
Level AI - Backend Engineer - Customer Engineering

Level AI

Noida, Uttar Pradesh, India (Hybrid)
4 Months ago
bosh group india - Technical Consultant

bosh group india

Bengaluru, Karnataka, India (On-Site)
2 Months ago
Assystems - Design Lead – Solar (Electrical DC)

Assystems

Gurugram, Haryana, India (On-Site)
3 Months ago
NVIDIA - Silicon Solutions Engineer - NPQ

NVIDIA

Bengaluru, Karnataka, India (Hybrid)
1 Month ago
Salesforce - Territory Account Executive

Salesforce

Gurugram, Haryana, India (On-Site)
1 Month ago
Head Digital Works - Sr. UX Manager

Head Digital Works

Hyderabad, Telangana, India (On-Site)
4 Months ago
bosh group india - MS/ECO-XC_Android_Automotive_Expert

bosh group india

Bengaluru, Karnataka, India (On-Site)
2 Months ago
InvenioLSI - SAP Finance Delivery Practice Lead

InvenioLSI

New Delhi, Delhi, India (On-Site)
2 Months ago
Sportskeeda - Gaming Writer-Deltias

Sportskeeda

India (Remote)
1 Month ago

Get notifed when new similar jobs are uploaded

Administrative Jobs

Saviynt - Technical Lead, Support Operations-  JVM Diagnostics

Saviynt

Bengaluru, Karnataka, India (Hybrid)
4 Months ago
Corsair - Sr. Payroll Specialist

Corsair

Taoyuan City, Taiwan (On-Site)
6 Days ago
InvenioLSI - Associate Manager, Resource Management

InvenioLSI

New Delhi, Delhi, India (On-Site)
1 Month ago
The Walt Disney Company - Technical Assistant

The Walt Disney Company

Mumbai, Maharashtra, India (On-Site)
2 Weeks ago
Saviynt - Consultant, Professional Services (EMEA Region)

Saviynt

Bengaluru, Karnataka, India (Hybrid)
4 Months ago
Nielsen Holdings - Senior SAP Payroll Technology Lead- Mumbai/Bangalore

Nielsen Holdings

Mumbai, Maharashtra, India (Hybrid)
2 Months ago
Alpha Sense - Principal People Systems Analyst

Alpha Sense

Ireland (Remote)
1 Month ago
The Walt Disney Company - Executive Assistant, Business Affairs

The Walt Disney Company

Burbank, California, United States (On-Site)
5 Days ago
Blinkhealth - Pharmacy Technician, Intake

Blinkhealth

St. Louis, Missouri, United States (On-Site)
1 Day ago

Get notifed when new similar jobs are uploaded

About The Company

PhonePe was founded in December 2015 and has emerged as India’s largest payments app, enabling digital inclusion for consumers and merchants alike. With 48 crore (480 Million) registered users, one in four Indians are now on PhonePe. The company has also successfully digitized 3.6 crore (36 Million) offline merchants spread across Tier 2,3,4 and beyond, covering 99% of the postal codes across India. PhonePe is also the leader in Bharat Bill Pay System (BBPS), processing over 45% of the transactions on the BBPS platform. PhonePe forayed into financial services in 2017, providing users with safe and convenient investing options on its platform. Since then, the company has introduced several Mutual Funds and Insurance products that offer every Indian an equal opportunity to unlock the flow of money and access to services. PhonePe was recently recognized as the Most Trusted Brand for Digital Payments as per the Brand Trust Report 2023 by Trust Research Advisory (TRA).



Pune, Maharashtra, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

Bengaluru, Karnataka, India (On-Site)

View All Jobs

Get notified when new jobs are added by Phonepe

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug