{"id":1515168,"url":"https://alion.io/job/hcltech-senior-apache-spark-technical-specialist-scala-python","title":"Senior Apache Spark Technical Specialist - Scala, Python","company":{"id":225,"name":"HCLTech","domain":"hcltech.com","url":"https://alion.io/company/hcltech","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SuccessFactors","truth_index":null},"role":"Backend","role_family":"Backend","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"Cassandra","optional":false},{"name":"CI/CD","optional":false},{"name":"Flink","optional":false},{"name":"HBase","optional":false},{"name":"Java","optional":false},{"name":"Rest API","optional":false},{"name":"Spring Boot","optional":false},{"name":"Power BI","optional":true},{"name":"SQL","optional":true},{"name":"Tableau","optional":true}],"status":"live","first_seen_at":"2026-09-30T09:21:50Z","employer_posted_date":"2026-09-30","last_verified_at":"2026-10-17T17:14:06Z","board_verified":true,"closed_at":null,"days_open":17,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":17},"description":"Job Summary\nWe are seeking a Senior Solution Architect with deep experience designing and delivering enterprise-scale big data, streaming, and real-time data integration solutions.\nThe role is focused on solution architecture across Cloudera-based big data platforms, Kafka-based streaming applications, and real-time data integration / analytics solutions, including cloud-hosted data platforms such as Cloudera on AWS or equivalent AWS-native services.\nThe role also provides architectural leadership across Data Solutions, helping shape architecture strategy, design standards, engineering best practices, and the future evolution of the Bank's data platform capabilities.\nKey Responsibilities\nOwn the end-to-end solution architecture for large-scale distributed data, streaming, and data integration solutions.\nAct as the lead architect for Data Solutions distributed data, integration and streaming platforms, providing architectural direction, governance, design assurance and technical leadership across initiatives, while influencing the evolution of the target architecture and ensuring alignment with enterprise technology strategy, engineering standards and long-term platform objectives.\nAccountable for solution architecture across the Enterprise Data Lake, Operational Datastore and associated data platform services within the Cloudera CDP environment on AWS.\nDesign and document scalable distributed data and streaming solutions across Cloudera, Kafka, API, and cloud platforms, including ingestion, processing, storage, and consumption patterns.\nProvide architectural oversight across solution delivery, ensuring consistency, quality and alignment with approved designs.\nWork with engineering teams to ensure solutions are deliverable, scalable, secure, supportable, and aligned to enterprise architecture principles.\nReview technical designs produced by engineering teams and ensure alignment with the approved solution architecture.\nIdentify technical risks, dependencies, assumptions, non-functional requirements, and design trade-offs.\nSupport delivery planning by helping break solutions into iterative delivery increments and sprint-level outcomes.\nRepresent solution designs at architecture review boards and governance forums, engaging effectively with business stakeholders, delivery teams, and enterprise architects.\nDevelop and promote architectural patterns, engineering standards and best practices across the data platform ecosystem.\nSkill Requirements\nThe successful candidate must demonstrate strong practical experience across many of the following areas:\nDistributed data platforms and data lake architecture on Cloudera CDH/CDP or equivalent platforms.\nEvent streaming and real-time integration technologies, including Kafka and related messaging patterns.\nLarge-scale data processing and transformation frameworks such as Spark, Flink, Hive, Impala or equivalent technologies.\nAPI and service-based integration architectures using technologies such as Java, Spring Boot, and REST services.\nNoSQL and distributed data storage technologies including HBase, Cassandra, or equivalent platforms.\nCloud-hosted data platforms and services, preferably on AWS.\nModern engineering practices including DevOps, CI/CD, observability, resilience, and operational support.\nOther Requirements\nDesirable Experience\nFinancial services or banking experience.\nExperience with regulated, secure, enterprise-scale data platforms.\nExperience presenting to architecture governance boards.\nExperience leading architecture communities, defining standards, or contributing to capability development within an architecture practice.\nExposure to enterprise data governance, lineage, metadata, and data quality tooling.\nExperience with cloud-native data services on AWS.\nCandidate Profile\nThis role is suited to a hands-on Senior Solution Architect who can design and guide the delivery of complex data and streaming solutions.\nThe ideal candidate will:\nBe an architect first, with strong hands-on engineering awareness and technical credibility.\nHave designed and delivered production-scale distributed data, streaming, and integration solutions.\nUnderstand both architecture-level trade-offs and engineering-level implementation constraints.\nBe comfortable challenging, reviewing, and improving technical designs.\nBe able to influence senior stakeholders, lead architectural decision-making, and engage deeply with engineering teams.\nHave practical experience delivering scalable, resilient, secure, real-time or near-real-time data platforms and promoting technical excellence across teams.\nImportant Clarification\nThis is not a role for candidates whose experience is primarily in:\nData analysis\nData modelling only\nBI reporting incl Power BI / Tableau reporting\nDashboard development\nSQL reporting\nData governance only\nBusiness analysis\nConceptual data architecture without implementation experience\nCandidates must have demonstrable experience in solution architecture for distributed big data platforms, streaming systems, event-driven integration, or real-time data applications.","description_format":"text","description_chars":5094,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","IT Outsourcing & Dedicated Teams","Systems Integrators"],"lifecycle":[{"event":"open","at":"2026-09-30T09:21:50Z"}],"visa":[],"liveness":{"score":60,"band":"ok","label":"Likely open","p_open":1,"p_active":0.799,"p_room":0.75,"age_days":16,"expected_fill_days":22,"reasons":["conf:0","velocity","win:late","comp:brand"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/hcltech-senior-apache-spark-technical-specialist-scala-python","json_url":"https://alion.io/job/hcltech-senior-apache-spark-technical-specialist-scala-python.json","meta":{"generated_at":"2026-10-17T19:07:34Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2287,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":225},"rest":"https://alion.io/mcp/rest/get_company?id=225"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhcltech-senior-apache-spark-technical-specialist-scala-python"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhcltech-senior-apache-spark-technical-specialist-scala-python"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhcltech-senior-apache-spark-technical-specialist-scala-python"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/hcltech-senior-apache-spark-technical-specialist-scala-python\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhcltech-senior-apache-spark-technical-specialist-scala-python"}]}