{"id":17517,"url":"https://alion.io/job/capco-poland-data-engineer-scala-spark-java-bng-pune","title":"Data Engineer with Kafka","company":{"id":1444,"name":"Capco","domain":"capco.com","url":"https://alion.io/company/capco-poland","size_band":"5000+","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"B","score":76,"open_postings":120,"ghost_share":0.017,"stale_share":0.9,"repost_share":0.017,"time_to_fill_p50_days":42,"computed_at":"2026-10-17T05:46:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India","Mumbai, India","Hyderabad, India","Chennai, India","Pune, India","Gurgaon, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":15500,"max_usd":37000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":16},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Java","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Agile","optional":true},{"name":"Airflow","optional":true},{"name":"AWS","optional":true},{"name":"Azure","optional":true},{"name":"CI/CD","optional":true},{"name":"GCP","optional":true},{"name":"Hadoop","optional":true}],"status":"live","first_seen_at":"2026-06-19T12:03:19Z","employer_posted_date":"2026-10-15","last_verified_at":"2026-10-17T12:44:11Z","board_verified":true,"closed_at":null,"days_open":120,"trust":{"level":"ok","repost_count":1,"flags":[],"days_open":119},"description":"About Us\n“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. \nWHY JOIN CAPCO?\nYou will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.\nMAKE AN IMPACT\nInnovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.\n#BEYOURSELFATWORK\nCapco has a tolerant, open culture that values diversity, inclusivity, and creativity.\nCAREER ADVANCEMENT\nWith no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.\nDIVERSITY & INCLUSION\nWe believe that diversity of people and perspective gives us a competitive advantage.\nMAKE AN IMPACT\nJob Title: Data Engineer with Kafka\nPosition: Data Engineer with Kafka\nExperience: 5+ Years\nWork Mode: Hybrid (Capco Office)\nKey Responsibilities\nDesign, develop, and optimize large-scale data processing applications using Scala, Apache Spark, and Java.\nBuild and maintain high-performance data pipelines capable of processing 80-90 million records daily with a focus on scalability, reliability, and efficiency.\nDevelop and integrate Native APIs and data services to support business-critical applications and analytics platforms.\nCollaborate with cross-functional teams to gather requirements and deliver robust data engineering solutions.\nOptimize Spark jobs, data workflows, and distributed computing processes to ensure high throughput and low latency.\nImplement best practices for data quality, monitoring, governance, and operational excellence.\nTroubleshoot and resolve performance bottlenecks across data processing and ingestion pipelines.\nParticipate in code reviews and contribute to engineering standards, architecture decisions, and continuous improvement initiatives.\nRequired Skills & Experience\nStrong hands-on experience in Scala, Apache Spark, and Java.\nProven experience building and supporting large-scale distributed data processing systems handling tens of millions of records daily.\nExpertise in developing and consuming Native APIs and microservices.\nStrong understanding of Spark architecture, performance tuning, partitioning, caching, and optimization techniques.\nExperience with data modeling, ETL/ELT processes, and large-scale batch and streaming data pipelines.\nSolid understanding of distributed systems, concurrency, and high-volume data processing.\nExperience with SQL and relational/non-relational databases.\nStrong debugging, analytical, and problem-solving skills.\nPreferred Qualifications\nExperience working in enterprise-scale data environments within Financial Services, Payments, or FinTech domains.\nExposure to cloud platforms (AWS, Azure, or GCP) and containerized deployments.\nFamiliarity with Kafka, Airflow, Hadoop ecosystem, or similar big data technologies.\nExperience with CI/CD pipelines, DevOps practices, and Agile methodologies.\nWhat We're Looking For\nEngineers passionate about solving complex data challenges at scale.\nProfessionals who can design highly performant systems capable of processing and transforming massive datasets efficiently.\nTeam players who thrive in fast-paced, collaborative environments and take ownership of end-to-end delivery.","description_format":"text","description_chars":3811,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["RegTech, AML & Compliance","IT Consulting & Digital Transformation","Management Consulting"],"lifecycle":[{"event":"open","at":"2026-06-19T12:03:19Z"},{"event":"close","at":"2026-09-25T09:08:31Z"},{"event":"reopen","at":"2026-10-07T13:25:53Z"}],"visa":[],"liveness":{"score":10,"band":"cold","label":"Long shot","p_open":1,"p_active":0.356,"p_room":0.28,"age_days":119,"expected_fill_days":42,"reasons":["conf:3","stale_co","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/capco-poland-data-engineer-scala-spark-java-bng-pune","json_url":"https://alion.io/job/capco-poland-data-engineer-scala-spark-java-bng-pune.json","meta":{"generated_at":"2026-10-17T14:20:00Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":19274,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1444},"rest":"https://alion.io/mcp/rest/get_company?id=1444"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcapco-poland-data-engineer-scala-spark-java-bng-pune"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcapco-poland-data-engineer-scala-spark-java-bng-pune"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcapco-poland-data-engineer-scala-spark-java-bng-pune"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/capco-poland-data-engineer-scala-spark-java-bng-pune\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcapco-poland-data-engineer-scala-spark-java-bng-pune"}]}