{"id":3204472,"url":"https://alion.io/job/epsilon-senior-data-engineer","title":"Senior Data Engineer","company":{"id":410,"name":"Epsilon","domain":"epsilon.com","url":"https://alion.io/company/epsilon","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":21000,"max_usd":46000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":75},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Airflow","optional":false},{"name":"AWS","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Docker","optional":false},{"name":"ElasticSearch","optional":false},{"name":"GCP","optional":false},{"name":"Grafana","optional":false},{"name":"Hadoop","optional":false},{"name":"Jenkins","optional":false},{"name":"Kibana","optional":false},{"name":"Kubernetes","optional":false},{"name":"Logstash","optional":false},{"name":"Machine Learning","optional":false},{"name":"Power BI","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Scrum","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Tableau","optional":false}],"status":"live","first_seen_at":"2026-10-24T11:36:30Z","employer_posted_date":null,"last_verified_at":"2026-10-24T11:36:30Z","board_verified":false,"closed_at":null,"days_open":6,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":6},"description":"The Product team forms the crux of our powerful platforms and helps connect millions of customers worldwide with the brands that matter most to them. This team of innovative problem solvers develops and builds products that position Epsilon as a differentiator, encouraging an open and balanced marketplace built on respect for individuals, where every brand interaction holds value. Our full-cycle product engineering and data teams chart the future and set new benchmarks for our products by using industry-standard methodologies and sophisticated capabilities in data, machine learning, and artificial intelligence. Driven by a passion for delivering smart end-to-end solutions, this team plays a key role in Epsilon's success story.\n\nResponsibilities: \nAs a Senior Data Engineer, you will be instrumental in implementing and optimising advanced data processing solutions using Scala, Spark, and Hadoop.\nYou will collaborate with cross-functional teams to deploy scalable big data solutions on our on-premises and cloud infrastructure.\nYour responsibilities will include building, scheduling, and maintaining complex workflows, performing data integration and transformation tasks, and ensuring data quality through rigorous testing\n\nRequirements: \nStrong written and verbal communication skills (in English) are required to facilitate work across multiple countries and time zones.\nGood understanding of Agile Methodologies - SCRUM.\nOver 5+ years in Scala programming and Apache Spark for developing scalable big data solutions on both on-premises and cloud platforms, such as AWS and GCP.\nSkilled in implementing scalable, fault-tolerant data pipelines with comprehensive monitoring and alerting systems.\nImplementing robust automated testing frameworks for big data jobs, ensuring comprehensive data integrity and performance validation.\nUse and implementation of tools such as Apache Airflow, Jenkins, or similar to automate complex testing workflows, including integration, regression, and performance testing.\nConducting extensive data validation checks using automated and manual testing techniques to ensure accuracy and consistency in large-scale datasets.\nDeep understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.\nHands-on experience with Python for infrastructure module development and strong SQL skills for handling large datasets.\nFamiliar with data warehousing concepts and technologies.\nExperienced with GIT or equivalent source control systems.\nProficient in developing and implementing unit test cases to ensure code quality and reliability and experienced in using integration testing frameworks to validate system interactions.\nEffective collaborator with stakeholders and teams to understand requirements and develop solutions.\nAbility to work within tight deadlines, prioritise tasks effectively, and perform under pressure.\nExperience in mentoring junior staff.\n\nAdvantageous to have experience on below: \nHands-on with Databricks for unified data analytics, including Databricks Notebooks, Delta Lake, and Catalogues.\nProficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualisation.\nStrong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision-making.\nExperience with data visualisation tools like Tableau, Power BI, or Grafana.\nFamiliarity with Docker for containerization and Kubernetes for orchestration.\nSkills\nApache Spark, HDFS, Hadoop, MapReduce, Scala, Spark, YARN","description_format":"text","description_chars":3557,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Marketing Automation","Email Marketing"],"lifecycle":[{"event":"open","at":"2026-10-24T11:36:30Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":5,"expected_fill_days":27,"reasons":["seen:5","velocity","win:early"],"computed_at":"2026-10-30T05:48:23Z"},"pay":null,"html_url":"https://alion.io/job/epsilon-senior-data-engineer","json_url":"https://alion.io/job/epsilon-senior-data-engineer.json","meta":{"generated_at":"2026-10-30T13:24:57Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":337,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-31T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":410},"rest":"https://alion.io/mcp/rest/get_company?id=410"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fepsilon-senior-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fepsilon-senior-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fepsilon-senior-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/epsilon-senior-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fepsilon-senior-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3204472}},{"tool":"search_jobs","arguments":{"company":"epsilon.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3204472},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}