Community

Home >

Jobs >

Data Engineer

United States (Remote)

Data Engineer

1 Month ago • 3 Years + • Data Analyst

About the job

5 skills required for this role

Add these skills to join the top 1% applicants for this job

azure

python

data-analytics

communication

cross-functional

Job Description

Varonis seeks an experienced Data Engineer to design, build, and maintain scalable data pipelines using modern architectures (Databricks, Azure Data Factory) for AI/ML/LLM initiatives. Responsibilities include ETL/ELT pipeline development, optimizing for performance and cost, collaborating with data scientists and cybersecurity researchers, and designing innovative prompts for an autonomous LLM-based labeling platform. The role requires proficiency in Python (PySpark a plus) and experience with large-scale data solutions. The successful candidate will possess strong analytical and problem-solving skills, and excel in a collaborative environment.

Must have:

3+ years data engineering experience
Cloud-based data solutions expertise
Python proficiency (PySpark a plus)
Databricks & Azure Data Factory experience
Large-scale data solutions implementation
Prompt engineering experience

Good to have:

Vector databases & embedding techniques
MLOps experience

Description

Position Overview:
We seek an experienced Data Engineer with expertise in modern data architectures, pipeline engineering, and cloud-based data ecosystems. In this role, you will design and build efficient, scalable data pipelines that enable robust data integration for AI, ML, SLM, LLM, and advanced analytics initiatives. You will also work closely with our data scientists and engineers to ensure high-quality, high-performance data flow and contribute to our data-driven culture.

Responsibilities:

Design, build, and maintain scalable ETL/ELT pipelines to integrate data from diverse sources, optimizing for performance and cost efficiency.
Leverage Databricks and other modern data platforms to manage, transform, and process data for ML and AI models, supporting both real-time and batch processing workflows.
Work with cross-functional teams to implement data solutions that support model training, monitoring, and production deployment.
Collaborate with a cybersecurity research team to understand emerging threats and develop solutions that leverage advanced data analytics.
Design and develop innovative prompts and instruction sets to enhance our autonomous LLM-based labeling platform.
Optimize prompts to generate high-quality, coherent, and contextually relevant responses.
Collaborate with software and data engineers to integrate ML/LLM techniques into production systems.

Requirements:

3+ years of experience in data engineering, including cloud-based data solutions
Proven expertise in implementing large-scale data solutions.
Proficiency in Python. PySpark is a plus.
Experience with cloud and big data technologies such as Databricks and Azure Data factory.
Experience with prompt engineering techniques.
Experience with vector databases and embedding techniques is a plus.
Experience with MLOps is a plus.
Strong analytical and problem-solving skills, with the ability to evaluate and interpret complex data.
Excellent communication and collaboration skills, with the ability to work effectively in a multidisciplinary team.
Proven track record of delivering high-quality results in a fast-paced and dynamic environment.

We invite you to check out our Instagram Page to gain further insight into the Varonis culture!

@VaronisLife

Varonis is an equal-opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, national origin, disability, veteran status, and other legally protected characteristics.

#Li-Remote

View Full Job Description

Upload your resume, increase your shortlisting chances by 80%