Hadoop & UNIX Shell Scripting Engineer (Data Management, Automation & Performance Tuning)

11 Hours ago • 5 Years +

Job Summary

Job Description

This role involves managing and optimizing the Hadoop ecosystem, developing automation utilities using UNIX Shell scripting, and troubleshooting complex issues. The engineer will collaborate with vendors for platform enhancements, supporting enterprise data processing, performance tuning, and cloud migration. Responsibilities include developing and maintaining scripts, serving as a Level 3 point of contact, performing performance tuning, conducting proof-of-concept tests, and collaborating with vendor support teams. The role also involves process analysis and automation of utilities.
Must have:
  • Proficiency in UNIX Shell scripting for automation.
  • In-depth understanding of Hadoop architecture and ecosystem components.
  • Experience with SQL querying and database systems.
  • Familiarity with Git and enterprise version control practices.
  • Working knowledge of DevOps and CI/CD tools and processes.
Good to have:
  • Experience with Python scripting for automation.
  • Knowledge of Java programming language.
  • Familiarity with cloud platforms (AWS, Azure, GCP).
  • Exposure to Hadoop vendor support and collaboration processes.

Job Details

Job Summary

Synechron is seeking a dedicated and technically skilled Hadoop Shell Scripting Engineer to manage and optimize our Hadoop ecosystem. The role involves developing automation utilities, troubleshooting complex issues, and collaborating with vendors for platform enhancements. Your expertise will directly support enterprise data processing, performance tuning, and cloud migration initiatives, ensuring reliable and efficient data infrastructure that aligns with organizational goals.

Software Requirements

Required Skills:

  • Strong proficiency in UNIX Shell scripting with hands-on experience in developing automation utilities
  • In-depth understanding of Hadoop architecture and ecosystem components (HDFS, Hive, Spark)
  • Experience with SQL querying and database systems
  • Familiarity with Git and enterprise version control practices
  • Working knowledge of DevOps and CI/CD tools and processes

Preferred Skills:

  • Experience with Python scripting for automation and utility development
  • Knowledge of Java programming language
  • Familiarity with cloud platforms (AWS, Azure, GCP) related to Hadoop ecosystem support
  • Exposure to Hadoop vendor support and collaboration processes (e.g., Cloudera)

Overall Responsibilities

  • Develop, maintain, and enhance scripts and utilities to automate Hadoop cluster management and data processing tasks
  • Serve as the Level 3 point of contact for issues related to Hadoop and Spark platforms
  • Perform performance tuning and capacity planning to support enterprise data workloads
  • Conduct proof-of-concept tests for emerging technologies and evaluate their suitability for cloud migration projects
  • Collaborate with vendor support teams and internal stakeholders for issue resolution, feature requests, and platform improvements
  • Review and validate all changes going into production to ensure stability and performance
  • Continuously analyze process inefficiencies and develop new automation utilities to enhance productivity
  • Assist in capacity management and performance monitoring activities

Technical Skills (By Category)

Programming Languages:

  • Essential: UNIX Shell scripting
  • Preferred: Python, Java

Databases & Data Management:

  • Essential: Knowledge of SQL querying and database systems
  • Preferred: Experience with Hive, HDFS

Cloud Technologies:

  • Preferred: Basic familiarity with cloud platforms for Hadoop ecosystem support and migration

Frameworks & Libraries:

  • Not specifically applicable; focus on scripting and platform tools

Development Tools & Methodologies:

  • Essential: Git, version control, DevOps practices, CI/CD pipelines
  • Preferred: Automation frameworks, monitoring tools

Security Protocols:

  • Not explicitly specified but familiarity with secure scripting and data access controls is advantageous

Experience Requirements

  • Minimum of 5+ years of hands-on experience working with Hadoop clusters and scripting in UNIX shell
  • Proven experience in managing enterprise Hadoop/Spark environments
  • Experience in performance tuning, capacity planning, and utility development
  • Exposure to cloud migrations or proof-of-concept evaluations is a plus
  • Background in data engineering or platform support roles preferred

Day-to-Day Activities

  • Develop and enhance UNIX shell scripts for Hadoop automation and utility management
  • Troubleshoot and resolve complex platform issues as the Level 3 point of contact
  • Work with application teams to optimize queries and data workflows
  • Engage with vendor support teams for platform issues and feature requests
  • Perform system performance reviews, capacity assessments, and tuning activities
  • Lead initiatives for process automation, efficiency improvement, and new technology evaluations
  • Document procedures, scripts, and platform configurations
  • Participate in team meetings, provide technical feedback, and collaborate across teams on platform health

Qualifications

Educational Requirements:

  • Bachelor's degree in Computer Science, Information Technology, or related field
  • Equivalent professional experience in data engineering, platform support, or Hadoop administration

Certifications (Preferred):

  • Certificates in Hadoop ecosystem, Linux scripting, or cloud platform certifications

Training & Professional Development:

  • Ongoing learning related to big data platforms, automation, and cloud migration

Professional Competencies

  • Strong analytical and troubleshooting skills
  • Excellent written and verbal communication skills
  • Proven ability to work independently with minimal supervision
  • Collaborative team player with a positive attitude
  • Ability to prioritize tasks effectively and resolve issues swiftly
  • Adaptability to evolving technologies and environments
  • Focus on quality, security, and process improvement

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT
 

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.


All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice

Similar Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Skill Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Jobs in Bengaluru, Karnataka, India

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

Similar Category Jobs

Looks like we're out of matches

Set up an alert and we'll send you similar jobs the moment they appear!

About The Company

At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies spanend-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and technology firms. Through research and development initiatives in ourFinLabswe develop solutions for modernization, from Artificial Intelligence and Blockchain to Data Science models, Digital Underwriting, mobile-first applications and more.Over the last20+ years, our company has been honored with multiple employer awards, recognizing our commitment to our talented teams. With top clients to boast about, Synechron has a global workforce of14,500+,and has58offices in 21 countrieswithin key global markets.For more information on the company, please visit ourwebsiteorLinkedIncommunity.

Bengaluru, Karnataka, India (On-Site)

Singapore (On-Site)

Pune, Maharashtra, India (On-Site)

Pittsburgh, Pennsylvania, United States (On-Site)

Pune, Maharashtra, India (On-Site)

Pune, Maharashtra, India (On-Site)

Pune, Maharashtra, India (On-Site)

Pune, Maharashtra, India (On-Site)

View All Jobs

Get notified when new jobs are added by Synechron

Level Up Your Career in Game Development!

Transform Your Passion into Profession with Our Comprehensive Courses for Aspiring Game Developers.

Job Common Plug