Python Python

Web Data Extraction Engineer (Scraping & Automation)

About the role

We’re looking for an engineer to build and maintain reliable, scalable web crawlers and scrapers for our clients. You’ll design extraction pipelines, handle dynamic and JavaScript-heavy sites, and keep data flowing as target sites change.

What you’ll do

• Build and maintain crawlers and scrapers for static and dynamic (JS-rendered) websites
• Extract and normalize data from HTML and the DOM, and from APIs found through network inspection
• Automate browser workflows with Playwright, Puppeteer or Selenium in headless environments
• Manage proxy rotation, sessions, cookies, headers and request throttling
• Handle anti-bot measures (rate limiting, fingerprinting, CAPTCHA challenges) responsibly and in line with site terms and applicable law
• Monitor scrapers, find breakages when sites change, and fix them quickly
• Write clean, tested, maintainable code and document data sources and logic

What you bring

• 3+ years in software engineering, test automation or data extraction
• Strong understanding of HTTP: methods, headers, status codes, cookies, sessions, auth
• Solid HTML/CSS/DOM knowledge: XPath, CSS selectors, parsing strategies
• Hands-on experience with Playwright and/or Selenium, including headless browsers
• Proficiency in Python (Scrapy, BeautifulSoup, httpx/requests) or Node.js
• Experience with proxies (residential/datacenter, rotation) and browser dev tools for network inspection
• Professional English for direct client communication

Nice to have

• Distributed crawling (queues, Scrapy Cluster, Celery, Kafka)
• Docker and cloud deployment (AWS/GCP)
• Data storage and pipelines (Postgres, MongoDB, S3)
• Familiarity with browser fingerprinting and stealth techniques
• Awareness of scraping compliance (robots.txt, ToS, GDPR/CCPA)

Javascript, Automation, selenium, HTML, Python, WebScraping, API Concepts