Data Engineer

As a Data Engineer at P99Soft, you will design, develop, and maintain robust data pipelines to support machine learning and AI applications. You'll collaborate with cross-functional teams to collect, process, and analyze data from various sources for AI model training, validation, and deployment. You'll work with large language models (LLM) and generative AI technologies (e.g., OpenAI GPT) to implement intelligent systems and applications and will be involved in optimizing and scaling AI/ML pipelines to handle vast amounts of structured and unstructured data efficiently. Additionally, you will support the deployment and monitoring of AI models into production environments, ensuring high performance and low-latency operations. The role also involves implementing best practices for data management, data quality, and data security in AI-driven systems, troubleshooting and resolving data-related issues, and contributing to the design of data architecture to support the integration of AI models and machine learning frameworks.

Must Have

• 2-5 years of experience as a Data Engineer or similar role.
• Experience with LLMs, OpenAI, and Generative AI frameworks.
• Proficiency in Python, SQL, and other data engineering languages.
• Experience with data processing frameworks and cloud platforms.
• Familiarity with machine learning models, model deployment, and API integration.

Good to Have

• Experience with OpenAI’s GPT models and API integrations.
• Familiarity with advanced AI techniques (reinforcement/unsupervised learning).
• Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines.
• Experience working with both structured and unstructured data.
• Exposure to advanced data analytics and visualization tools (Tableau, Power BI).

Job Title: Data Engineer

Company: P99Soft Pvt Ltd

Location: Hyderabad/Bangalore/Pune

Job Type: Full-time

About Us:

P99Soft Pvt Ltd is an innovative tech company at the forefront of AI-driven solutions, empowering businesses with cutting-edge technologies. We specialize in delivering data-driven insights, and our mission is to build scalable, intelligent systems that solve real-world challenges. We are looking for a passionate and skilled Data Engineer to join our growing team and help shape the future of data science and AI.

Role Overview:

We are seeking a highly motivated and hands-on Data Engineer with expertise in large language models (LLM), OpenAI, and Generative AI (GenAI) technologies. This role requires a strong foundation in data engineering practices, alongside deep familiarity with machine learning, AI model integration, and processing large datasets for advanced AI-driven applications.

As a Data Engineer at P99Soft, you will work closely with data scientists, software engineers, and AI researchers to build scalable data infrastructure and deploy cutting-edge AI models, including OpenAI's tools and custom Generative AI solutions.

Key Responsibilities:

• Design, develop, and maintain robust data pipelines to support machine learning and AI applications, ensuring data integrity and efficiency.
• Collaborate with cross-functional teams to collect, process, and analyze data from various sources for AI model training, validation, and deployment.
• Work with large language models (LLM) and generative AI technologies (e.g., OpenAI GPT) to implement intelligent systems and applications.
• Optimize and scale AI/ML pipelines to handle vast amounts of structured and unstructured data efficiently.
• Support the deployment and monitoring of AI models into production environments, ensuring high performance and low-latency operations.
• Implement best practices for data management, data quality, and data security in AI-driven systems.
• Troubleshoot and resolve data-related issues and bottlenecks across the AI infrastructure.
• Contribute to the design of data architecture to support the integration of AI models and machine learning frameworks.

Key Requirements:

• 2-5 years of hands-on experience as a Data Engineer or in a similar role.
• Proven experience working with large language models (LLM), OpenAI technologies, and Generative AI (GenAI) frameworks.
• Strong proficiency in Python, SQL, and other programming languages commonly used in data engineering.
• Experience with data processing frameworks (e.g., Apache Spark, Apache Kafka, Hadoop) and cloud platforms (e.g., AWS, GCP, Azure).
• Familiarity with machine learning models, model deployment, and API integration.
• Solid understanding of data warehousing, ETL processes, and data architecture.
• Knowledge of data security, privacy standards, and compliance regulations.
• Strong problem-solving skills and ability to work in a fast-paced, collaborative environment.
• Excellent communication skills to work with cross-functional teams and stakeholders.

Preferred Skills:

• Experience with OpenAI’s GPT models and API integrations.
• Familiarity with advanced AI techniques like reinforcement learning, unsupervised learning, etc.
• Knowledge of containerization (e.g., Docker, Kubernetes) and CI/CD pipelines.
• Experience working with both structured and unstructured data (e.g., text, images, audio).
• Exposure to advanced data analytics and visualization tools (e.g., Tableau, Power BI, etc.).