As our Crawler Analyst, you will work using our internal tool to automate information extraction from websites. Every site has its own challenges so you’ll have to collaborate with the Bots (Python) team (and occasionally other teams and departments) to solve them as you learn, as well as with the developers of this tool to propose improvements. You also have to check system logs to detect possible errors in production and deduce if something is not well defined in terms of development.

Responsibilities

• Automate information extraction for hundreds of websites using custom templates.
• Keep extraction procedures working as websites change by updating templates (YAML files)
• Validate current templates and escalate the more complex ones to the Python development team.
• Thinking about elegant, reliable, and long term solutions.
• Provide feedback about the tool so that we can improve it and make the job easier over time.

Your qualifications:

• Have strong analytical and technical problem-solving skills.
• Have strong communication and organizational skills.
• Be able to work well in a team and autonomously.
• Be flexible and able to adapt easily to changes and learn quickly.
• Be able to learn new skills and tools using documentation and examples.
• Be proactive and propose solutions to detected problems, and think about how to improve the process.

Requirements:

• Experience in scraping, scripting, programming or coding is required, especially good knowledge of HTML, XPATH and regex.
• Experience working with Linux environments (grep, awk…)
• Basic knowledge of using Docker or similar tools.
• Familiarity with Agile methodologies (Scrum, kanban, Jira Suite...)
• Some familiarity with Python, although you will not use it frequently.
• Knowledge of Git as a version control system.
• Upper-Intermediate level of English (we are an international team and the official language in the office is English) 

Nice to have:

• More experience in Python will help you to solve and detect problems easily with our tool.
• Some experience working with Proxies.
• Ability to find solutions by observing logs.