Cloudera Developer
T systems
Job Summary
As a Sr Cloudera Developer (Data Engineer), you will be responsible for designing, developing, and implementing data solutions using Cloudera technologies like Hadoop, Spark, and Hive. You will collaborate with data engineers to optimize data pipelines and workflows, and work closely with data analysts and scientists to ensure data quality. You will troubleshoot issues and stay updated on the latest trends, participating in code reviews. Proficiency in Spark, ETL processes, and experience with Hadoop ecosystem tools are required, along with strong programming skills in Java, Scala, or Python. Excellent problem-solving and communication skills are also necessary.
Must Have
- Proficiency in working with Spark
- Experience in optimizing Spark execution plan
- Skills in performing ETL processes using Spark
- Strong knowledge of Python
- Experience with data modeling and integration
- Solid understanding of Cloudera technologies
Good to Have
- Experience with integrating Spark Streaming with other technologies
- Familiarity with the Hadoop ecosystem
- Experience with deploying and managing Spark applications
- Experience in GCP services
Job Description
Company Description
T-Systems Information and Communication Technology India Private Limited (T-Systems ICT India Pvt. Ltd.) is a proud recipient of the prestigious Great Place To Work® Certification™. As a wholly owned subsidiary of T-Systems International GmbH, T-Systems India operates across Pune, Bangalore, and Nagpur, boasting a dedicated team of 3500+ employees providing services to group customers. T-Systems offers integrated end-to-end IT solutions, driving the digital transformation of companies in all industries, including automotive, manufacturing, logistics, and transportation, as well as healthcare and the public sector. T-Systems develops vertical, company-specific software solutions for these sectors. T-Systems International GmbH is an information technology and digital transformation company with a presence in over 20 countries and a revenue of more than €4 billion. T-Systems is a world-leading provider of digital services and has over 20 years of experience in the transformation and management of IT systems. As a subsidiary of Deutsche Telekom and a market leader in Germany, T-Systems International offers secure, integrated information technology and digital solutions from a single source.
Job Description
Role : Sr Cloudera Developer (Data Engineer)
Exp : 6 to 10 Years
Location : Pune
Job Description :
- Proficiency in working with Spark
- Understanding of Spark s architecture and fault tolerance mechanisms
- Proficiency in using Spark DataFrames and Spark SQL for querying structured data
- Experience in optimizing Spark execution plan is a plus
- Skills in performing Extract Transform and Load ETL processes using Spark
- Experience with integrating Spark Streaming with other technologies like Kafka is an advantage
- Familiarity with the Hadoop ecosystem including tools such as HDFS Hive Cloudera stack can be of advantage
- Experience with deploying and managing Spark applications on a Hadoop cluster or on GCP Dataproc
- Strong knowledge of Python experience with Java is beneficial as well
- DevOps tools and practices CI CD Docker
- Hands on experience in GCP services Dataproc Cloud Function Cloud Run Pub Sub BigQuery
Responsibilities and Duties :
• Design, develop, and implement data solutions using Cloudera technologies such as Hadoop, Spark, and Hive
• Collaborate with data engineers to optimize data pipelines and data processing workflows.
• Work closely with data analysts and data scientists to ensure data quality and integrity.
• Troubleshoot and resolve issues with data processing and data storage systems.
• Stay up-to-date on the latest trends and best practices in Cloudera development
• Participate in code reviews and provide feedback to team members.
Qualifications and Skills:
• Bachelor’s degree in computer science, Information Technology, or a related field
• Proven experience as a Cloudera Developer or similar role
• Solid understanding of Cloudera technologies such as Hadoop, Spark, and Hive
• Experience with data modeling, data warehousing, and data integration.
• Strong programming skills in Java, Scala, or Python
• Excellent problem-solving and communication skills
• Ability to work independently and as part of a team.