Python Web Developer - Web Scraping & Crawling
First seen by Alion on Nov 4, 2026.
Job Summary:
We are looking for 4 experienced Python Web Developers to support our Content Factory program. The selected resources will be responsible for extracting and collecting data from customer websites that do not provide bulk data download capabilities.
Developers will leverage the platform provided by the client to build and maintain web crawlers and scraping solutions that automate website data extraction.
Candidates should have strong Python programming skills and general web development experience. Prior experience developing web crawlers or web scraping solutions is preferred but not mandatory.
Key Responsibilities:
- Develop and implement web crawlers and scraping scripts using Python.
- Extract structured data from customer websites that do not support bulk data downloads.
- Analyze website structures, HTML pages, and web elements to identify relevant data.
- Utilize Python libraries and frameworks for automated data extraction.
- Clean, validate, and organize scraped data to ensure accuracy and consistency.
- Troubleshoot scraping issues and handle changes in website structures.
- Work with the client-provided platform to build and execute data extraction workflows.
- Collaborate with the team to meet project timelines and data delivery requirements.
Required Skills:
- Strong hands-on experience with Python programming.
- General web development experience.
- Good understanding of HTML, CSS, and JavaScript fundamentals.
- Experience working with REST APIs and JSON.
- Familiarity with HTTP requests, web page structures, and DOM elements.
- Basic understanding of databases and data handling.
- Strong debugging and problem-solving skills.
Preferred Skills:
- Experience with web scraping and web crawling.
- Familiarity with Python libraries such as BeautifulSoup, Requests, Scrapy, and Selenium.
- Knowledge of browser automation tools such as Playwright.
- Understanding of XPath, CSS selectors, and HTML parsing.
- Experience handling dynamic websites and JavaScript-rendered content.
Candidate Profile:
- Developers with 6 - 9 years of relevant development experience.
- Strong Python programming and web development fundamentals.
- Ability to independently develop and troubleshoot scraping scripts.
- Practical, hands-on development experience.
- Prior crawler development experience is an added advantage, but not mandatory.
Skills
Python, Web Scraping, RESTful API, JSON, Scrapy, Selenium, Playwright Testing, Web Crawling