About the job
Equal Opportunity Employer
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets
Job Description :
Senior Data Engineer
One of the most innovative and high-profile teams in News Corp is looking for a seasoned data engineer to help accelerate its vision. The role will involve combining all of News Corp’s data across categories, countries and organizations into a singular view of a customer enabling engagement, measurement and targeting. The HQData Senior Data Engineer will partner closely with product, data & engineering teams to help make this data available and accessible to our businesses, teams and partners in order to drive revenue and improve the customer experience.
Responsibilities
- Design, implement and maintain high performance big data infrastructure/systems & big data processing pipelines scaling to billions of structured and unstructured events daily
- Design, implement and maintain deep integration with up-stream systems
- Monitor performance of the data platform and optimize as needed
- Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
- Support product with the overall roadmap and ensure updates to senior leadership are 100% technically correct.
- Data analysis, understanding of business requirements and translation into logical pipelines & processes
- Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)
- Evaluate and prototype new technologies in the area of data processing
- Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
- High energy level, strong team player and good work ethic
Technologies we use
- Google Cloud Platform (Storage, Dataflow, BigQuery, DataStudio, DataTransfer, Composer)
- Python, R, Spark (PySpark)
- Kubernetes
- Airflow
- Git for source code management
- Jira
Qualifications
- BS in Computer Science or other technical discipline
- 5+ years of experience in designing and developing big data processing systems using distributed computing
- 5+ years in Python Programming
- 5+ years experience in SQL
- 4+ years experience in Apache Airflow
- Working with REST APIs
- 5+ years of Hands on experience in Google Cloud Platform
- Expert knowledge in optimizing complex SQL queries
- Experience with job orchestration tools
- An affinity for automation
- Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs
- Experience with at least one object oriented language (ex: Java)
- Strong OO design, data structure, and algorithm design skills
This job is posted with NTS Technology Services Pvt. Ltd.
Job Category:
