{"id":680669,"url":"https://alion.io/job/portpro-director-of-software-engineering-node-js-web-scraping-expert","title":"Director of Software Engineering (Node.js & Web Scraping Expert)","company":{"id":685453,"name":"PortPro","domain":"portpro.io","url":"https://alion.io/company/portpro","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"B","score":75,"open_postings":5,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-30T05:48:23Z"}},"role":"Leadership","role_family":"Leadership","seniority":"head","employment_type":null,"work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Los Angeles, United States"],"countries":["US"],"hiring_countries":["US"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":162000,"max_usd":291000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":727},"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Cheerio","optional":false},{"name":"CI/CD","optional":false},{"name":"Docker","optional":false},{"name":"ElasticSearch","optional":false},{"name":"GCP","optional":false},{"name":"GDPR","optional":false},{"name":"Kanban","optional":false},{"name":"Kubernetes","optional":false},{"name":"Node JS","optional":false},{"name":"Playwright","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Puppeteer","optional":false},{"name":"RabbitMQ","optional":false},{"name":"Redis","optional":false},{"name":"Scrum","optional":false},{"name":"Selenium","optional":false},{"name":"SQL","optional":false},{"name":"JavaScript","optional":true},{"name":"Machine Learning","optional":true},{"name":"NLP","optional":true}],"status":"live","first_seen_at":"2025-03-11T12:35:38Z","employer_posted_date":"2025-03-11","last_verified_at":"2026-10-30T11:38:43Z","board_verified":true,"closed_at":null,"days_open":598,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":596},"description":"We are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping. This role will lead the engineering team, designing and optimizing high-performance, distributed web scraping systems. The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures.\nKey Responsibilities- Software Engineering & Web Scraping Leadership:\nArchitect, develop, and maintain scalable and distributed web scraping systems using Node.js.\nDesign and implement data extraction pipelines to process large volumes of structured and unstructured data.\nDevelop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotation.\nOptimize scraping processes for performance, reliability, and efficiency while managing proxy services (residential, datacenter, rotating).Oversee data storage and processing strategies, ensuring high availability and consistency.\nCollaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications.\nImplement best practices for microservices, API integrations, and real-time data streaming.\nKey Responsibilities- Scalability, Security & DevOps:\nLead the transition to cloud-native, containerized, and serverless architectures for web scraping.\nEnsure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.).Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scraping.\nManage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecks.\nWork closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalability.\nKey Repsonsibilities- Engineering Team Management & Strategy:\nLead, mentor, and grow a high-performance engineering team.\nDefine and execute the technology roadmap, aligning with business objectives.\nFoster a culture of continuous learning, collaboration, and innovation.\nImplement agile development methodologies (Scrum, Kanban) to optimize project execution.\nEnsure code quality, security, and best practices across all engineering efforts.\nQualifications & Experience- Technical Expertise:\n10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extraction.\nStrong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation.\nExtensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques.\nDeep knowledge of proxy management (residential, datacenter, rotating, and VPNs).Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.).\nFamiliarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.).Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure).\nQualifications & Experience- Leadership & Soft Skills:\nProven track record of scaling engineering teams and leading complex projects.\nStrong problem-solving and debugging skills, especially for scraping challenges and performance bottlenecks.\nExcellent communication and stakeholder management skills.\nPassion for mentorship, team development, and continuous learning.\nPreferred Qualifications:\nExperience with machine learning for data extraction and NLP.\nKnowledge of browser fingerprinting and bot detection mechanisms.\nFamiliarity with enterprise-scale web crawling frameworks (Scrapy, Colly, Apify, etc.).\nPrior leadership experience in data-driven businesses or web scraping startups.","description_format":"text","description_chars":3664,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Continuous learning"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Logistics","Supply Chain & Logistics Software"],"lifecycle":[{"event":"open","at":"2026-09-10T23:27:07Z"}],"visa":[],"liveness":{"score":8,"band":"cold","label":"Long shot","p_open":1,"p_active":0.29,"p_room":0.28,"age_days":597,"expected_fill_days":30,"reasons":["conf:2","win:tail","crowd:"],"computed_at":"2026-10-30T05:48:23Z"},"pay":null,"html_url":"https://alion.io/job/portpro-director-of-software-engineering-node-js-web-scraping-expert","json_url":"https://alion.io/job/portpro-director-of-software-engineering-node-js-web-scraping-expert.json","meta":{"generated_at":"2026-10-30T15:13:19Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2967,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-31T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":685453},"rest":"https://alion.io/mcp/rest/get_company?id=685453"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fportpro-director-of-software-engineering-node-js-web-scraping-expert"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fportpro-director-of-software-engineering-node-js-web-scraping-expert"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fportpro-director-of-software-engineering-node-js-web-scraping-expert"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/portpro-director-of-software-engineering-node-js-web-scraping-expert\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fportpro-director-of-software-engineering-node-js-web-scraping-expert"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":680669}},{"tool":"search_jobs","arguments":{"company":"portpro.io","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":680669},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}