Data Engineer

About the Organization Now is a great time to join Redhorse Corporation. Redhorse specializes in developing and implementing creative strategies and solutions with private, state, and federal customers in the areas of cultural and environmental resources services, climate and energy change, information technology, and intelligence services. We are hiring creative, motivated, and talented people with a passion for doing what's right, what's smart, and what works.
About the role Redhorse transforms how government uses data and technology. We are seeking a skilled Data Engineer to play a pivotal role in designing, building, and maintaining a cutting-edge cloud-based data environment for a critical intelligence Sponsor. This role is crucial for enabling analysts to access and utilize large datasets to address high-priority intelligence questions. You will be a key contributor to a high-impact mission, directly influencing the Sponsor’s analytical capabilities and decision-making processes.
Key Responsibilities

• Assist with strategic planning and oversee the implementation of the Sponsor's cloud-based data environment.
• Develop and maintain ETL (Extract, Transform, Load) processes for efficient data movement and transformation.
• Design and implement data models and access controls to ensure data integrity and security.
• Develop code (Python, Bash, etc.), documentation, and data models that adhere to Sponsor standards.
• Provide systems administration and programming support for ETL processes and data infrastructure.
• Train and conduct knowledge transfer to team members on ETL processes, the on-premise compute cluster, and administrative duties.
• Coordinate with external data and platform providers to ensure smooth system functioning and data flows.
• Support the cross-domain transfer and integration of data.
• Serve as a technical liaison between engineers, data scientists, analysts, and managers.

Required Experience/Clearance

• 5+ years of experience as a Data Engineer.
• Demonstrated experience with AWS cloud services, including long-term storage options, and cloud-based database services (e.g., Databricks or EMR).
• Proficiency in SQL database structures and mapping between SQL databases.
• Experience with large-scale data migration efforts.
• Experience with database architecture, performance design methodologies, and system-tuning recommendations.
• Proficiency in Python and Bash scripting.
• Experience implementing CI/CD pipelines using industry-standard processes.
• TS/SCI with poly required to start

Desired Experience

• Experience with the Sponsor’s data environment and on-premises compute structure.
• Experience with Glue, Hive, and Iceberg or similar technologies.
• Experience with Terraform.
• Experience with DevSecOps solutions and tools.
• Experience with Data Quality and Data Governance concepts and experience.
• Experience maintaining, supporting, and improving the ETL process using Apache NiFi or similar tools.
• Experience with Apache Spark.

Equal Opportunity Employer/Veterans/Disabled    Accommodations: If you are a qualified individual with a disability or a disabled veteran, you may request a reasonable accommodation if you are unable or limited in your ability to access job openings or apply for a job on this site as a result of your disability. You can request reasonable accommodations by contacting Talent Acquisition at Talent-Acquisition@redhorsecorp.com   Redhorse Corporation shall, in its discretion, modify or adjust the position to meet Redhorse’s changing needs. This job description is not a contract and may be adjusted as deemed appropriate in Redhorse’s sole discretion.