{"id":3672631,"url":"https://alion.io/job/capco-big-data-engineer-scala","title":"Big Data Engineer- Scala","company":{"id":1444,"name":"Capco","domain":"capco.com","url":"https://alion.io/company/capco-poland","size_band":"5000+","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"B","score":77,"open_postings":129,"ghost_share":0.023,"stale_share":0.845,"repost_share":0.039,"time_to_fill_p50_days":42,"computed_at":"2026-11-05T05:54:30Z"}},"role":"Data Science","role_family":"Data Science","seniority":null,"employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"CI/CD","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"Hadoop","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Agile","optional":true},{"name":"Airflow","optional":true},{"name":"AWS","optional":true},{"name":"Azure","optional":true},{"name":"Databricks","optional":true},{"name":"Delta Lake","optional":true},{"name":"Docker","optional":true},{"name":"GCP","optional":true},{"name":"Kubernetes","optional":true},{"name":"Scrum","optional":true}],"status":"live","first_seen_at":"2026-11-03T06:06:03Z","employer_posted_date":"2026-11-03","last_verified_at":"2026-11-05T05:33:16Z","board_verified":true,"closed_at":null,"days_open":2,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":2},"description":"Job Title: Big Data Engineer - SCala\nAbout Us\n“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 byAvtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. \nWHY JOIN CAPCO?\nYou will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.\nMAKE AN IMPACT\nInnovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.\n#BEYOURSELFATWORK\nCapco has a tolerant, open culture that values diversity, inclusivity, and creativity.\nCAREER ADVANCEMENT\nWith no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.\nDIVERSITY & INCLUSION\nWe believe that diversity of people and perspective gives us a competitive advantage.\nMAKE AN IMPACT\n\nJob Description: Big Data Engineer - Scala\nWe are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data technologies.\nKey Responsibilities\nDesign, develop, and maintain scalable Big Data applications and data pipelines.\nDevelop high-performance data-processing solutions using Scala and Apache Spark.\nBuild and optimize ETL/ELT pipelines for large volumes of structured and unstructured data.\nWork with technologies such as Hadoop, HDFS, Hive, Kafka, and Spark SQL.\nPerform Spark job performance tuning and optimization.\nDevelop reusable and maintainable data-processing frameworks.\nTroubleshoot production issues related to data pipelines and distributed applications.\nImplement data-quality checks, monitoring, and error-handling mechanisms.\nCollaborate with data architects, analysts, business teams, and other engineering teams.\nFollow software engineering best practices, including Git, code reviews, testing, and CI/CD.\nRequired Skills\nStrong hands-on programming experience in Scala.\nStrong experience with Apache Spark / Spark SQL.\nGood understanding of Big Data and distributed computing concepts.\nExperience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive.\nExperience building large-scale ETL/data engineering pipelines.\nStrong SQL skills.\nExperience working with Kafka or similar messaging/streaming technologies.\nUnderstanding of data formats such as Parquet, Avro, and JSON.\nExperience with performance optimization and troubleshooting of Spark applications.\nGood understanding of software development lifecycle and version-control systems.\nGood to Have\nExperience with AWS, Azure, or GCP.\nKnowledge of Databricks.\nExperience with Airflow or similar orchestration tools.\nExposure to Delta Lake or other modern lakehouse technologies.\nKnowledge of Docker, Kubernetes, and CI/CD pipelines.\nExperience working in Agile/Scrum environments.\nCore Skills\nScala | Apache Spark | Spark SQL | SQL | Hadoop | HDFS | Hive | Kafka | ETL | Data Engineering | Big Data","description_format":"text","description_chars":3564,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["RegTech, AML & Compliance","IT Consulting & Digital Transformation","Management Consulting"],"lifecycle":[{"event":"open","at":"2026-11-03T09:50:21Z"}],"visa":[],"liveness":{"score":63,"band":"ok","label":"Likely open","p_open":1,"p_active":0.632,"p_room":1,"age_days":1,"expected_fill_days":42,"reasons":["conf:0","stale_co","velocity","win:early","comp:brand"],"computed_at":"2026-11-05T05:54:30Z"},"pay":null,"html_url":"https://alion.io/job/capco-big-data-engineer-scala","json_url":"https://alion.io/job/capco-big-data-engineer-scala.json","meta":{"generated_at":"2026-11-05T09:52:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":12564,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-06T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1444},"rest":"https://alion.io/mcp/rest/get_company?id=1444"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcapco-big-data-engineer-scala"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcapco-big-data-engineer-scala"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcapco-big-data-engineer-scala"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/capco-big-data-engineer-scala\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcapco-big-data-engineer-scala"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3672631}},{"tool":"search_jobs","arguments":{"company":"capco.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3672631},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}