{"id":3290129,"url":"https://alion.io/job/demandmatrix-big-data-lead-2","title":"Big Data Lead","company":{"id":698524,"name":"DemandMatrix","domain":"demandmatrix.com","url":"https://alion.io/company/demandmatrix","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SmartRecruiters","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"inferred_company_offices","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["IN"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":110000,"max_usd":228000,"period":"year","method":"global_role_seniority_cell","sample_n":771},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"ElasticSearch","optional":false},{"name":"GCP","optional":false},{"name":"Hadoop","optional":false},{"name":"HBase","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"Unix","optional":false},{"name":"CI/CD","optional":true},{"name":"Docker","optional":true},{"name":"Kubernetes","optional":true},{"name":"Machine Learning","optional":true}],"status":"live","first_seen_at":"2026-09-22T10:27:24Z","employer_posted_date":null,"last_verified_at":"2026-09-22T10:27:24Z","board_verified":false,"closed_at":null,"days_open":38,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":38},"description":"What will you do?\nTo help us go to the next level we are looking to onboard a hands-on SME in leveraging big data tech to solve the most complex data issues. You will spend almost half of time with hands-on coding.\n\nIt involves large scale text data processing, event driven data pipelines, in-memory computations, optimization considering CPU core to network IO to disk IO.\n\nYou will be using cloud native services in AWS and GCP.\n\nWho Are You? \nSolid grounding in computer engineering, Unix, data structures and algorithms would enable you to meet this challenge.\n\nDesigned and built multiple big data modules and data pipelines to process large volume. \n\nGenuinely excited about technology and worked on projects from scratch. \n\nMust have:\n7+ years of hands-on experience in Software Development with a focus on big data and large data pipelines.\n\nMinimum 3 years of experience to build services and pipelines using Python.\n\nExpertise with a variety of data processing systems, including streaming, event, and batch (Spark, Hadoop/MapReduce)\n\nUnderstanding of at least one NoSQL stores like MongoDB, Elasticsearch, HBase\n\nUnderstanding of how data models, sharding and data location strategies for distributed data stores in large scale high-throughput and high-availability environments and their effect in non-structured text data processing\n\nExperience with running scalable & high available systems with AWS or GCP.\n\nGood to have:\nExperience with Docker / Kubernetes\n\nExposure with CI/CD\n\nKnowledge of Crawling/Scraping\n\nEntire Work From Home\n\nBirthday Leave\n\nRemote Work \n\nAt DemandMatrix, our vision is to disrupt the $100 billion sales and marketing intelligence industry by using domain knowledge, machine learning and AI. Fortune 100 companies like Microsoft, Google, Adobe, Amazon, IBM trust us to identify their next customer.\nOriginally posted on Himalayas","description_format":"text","description_chars":1871,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Data Providers & Datasets","B2B Company & Contact Data"],"lifecycle":[{"event":"open","at":"2026-10-26T19:11:04Z"}],"visa":[],"liveness":{"score":23,"band":"cold","label":"Long shot","p_open":0.6,"p_active":0.697,"p_room":0.55,"age_days":37,"expected_fill_days":33,"reasons":["seen:37","win:tail"],"computed_at":"2026-10-30T05:48:23Z"},"pay":null,"html_url":"https://alion.io/job/demandmatrix-big-data-lead-2","json_url":"https://alion.io/job/demandmatrix-big-data-lead-2.json","meta":{"generated_at":"2026-10-31T05:37:30Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":8152,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-01T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":698524},"rest":"https://alion.io/mcp/rest/get_company?id=698524"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fdemandmatrix-big-data-lead-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fdemandmatrix-big-data-lead-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fdemandmatrix-big-data-lead-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/demandmatrix-big-data-lead-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fdemandmatrix-big-data-lead-2"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3290129}},{"tool":"search_jobs","arguments":{"company":"demandmatrix.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3290129},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}