Middle Scraping Engineer

This is not a generic backend role. You will work with complex web structures, heavy anti-bot mechanisms, scale-driven infrastructure, and high-load scraping systems.

You will expand our crawling coverage, maintain a massive crawling fleet, architect unblocking solutions, and ensure high data quality under strict SLAs.

Why this role is exciting

Scraping at the Middle level is advanced web reverse engineering combined with scalable systems architecture.

Every target site is a dynamic battlefield: advanced anti-bots, CAPTCHAs, fingerprinting, dynamic JavaScript rendering, and changing structures. You won't just write scripts - you will build resilient, automated data-pipeline infrastructure that handles millions of requests seamlessly.

We offer an environment where your technical initiative leads directly to production impact. You will have full ownership of challenging scrapers and the freedom to experiment with new unblocking and engineering approaches.

What you will do

• Maintain and optimize our existing large-scale crawling fleet to ensure high availability;
• Expand crawling coverage by developing scrapers for new, complex target websites;
• Improve the quality, throughput, and success rates of existing crawls;
• Research, update, and develop custom unblocking solutions to bypass modern anti-bot protections;
• Ensure consistent data delivery while strictly maintaining client SLAs;
• Support team bandwidth by actively taking on R&D tasks, solving complex edge cases, and mentoring junior engineers.

• Strong Python mastery ("Python Ninja") with clean, scalable code practices;
• Deep expertise in Scrapy, Playwright, and async networking;
• Solid hands-on experience with proxy management (residential, datacenter, rotating networks, session stickiness);
• Experience with advanced unblocking tools and techniques (bypassing Cloudflare, Akamai, Datadome, CAPTCHA solvers, TLS/browser fingerprinting);
• Practical DevOps knowledge (Docker, CI/CD pipelines, container orchestration, monitoring);
• Strong web reverse engineering skills (analyzing network traffic, API endpoints, WebSocket messages, minified JS);
• Experience in building or deploying scalable web scraping infrastructure;
• High level of responsibility, SLA-driven mindset, and target-oriented approach;
• Excellent teamwork, fast thinking, and strong problem-solving skills in high-pressure situations.

• Rich innovative software stack, freedom to choose the best suitable technologies;
• Remote-first ideology: freedom to operate from the home office or any suitable coworking;
• Flexible working hours (we start from 8 to 11 am) and no time tracking systems on;
• Regular performance and compensation reviews;
• Recurrent 1-1s and measurable OKRs;
• In-depth onboarding with a clear success track;
• Competera covers 70% of your training/course fee;
• 20 vacation days, 15 days off, and up to one week of paid Christmas holidays;
• 20 business days of sick leave.