{"id":2123929,"url":"https://alion.io/job/epam-systems-poland-seniorlead-data-software-engineer-python-spark-azure","title":"Senior/Lead Data Software Engineer (Python, Spark, Azure)","company":{"id":2612641,"name":"EPAM Systems Poland","domain":"epam-poland.pl","url":"https://alion.io/company/epam-poland","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Backend","role_family":"Backend","seniority":"lead","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Kraków, Poland"],"countries":["PL"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":66000,"max_usd":108000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":59},"experience_years_min":3,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Anthropic","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Azure AKS","optional":false},{"name":"Databricks","optional":false},{"name":"Docker","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"GitLab CI","optional":false},{"name":"Java","optional":false},{"name":"Kubernetes","optional":false},{"name":"Machine Learning","optional":false},{"name":"Platform Engineering","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"Terraform","optional":false}],"status":"live","first_seen_at":"2026-10-07T11:05:05Z","employer_posted_date":null,"last_verified_at":"2026-10-07T11:05:05Z","board_verified":false,"closed_at":null,"days_open":24,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":24},"description":"We are seeking a Senior/Lead Data Software Engineer to join our team working on a scalable, ML-ready platform that enhances portfolio model development and deployment with advanced data governance and AI capabilities.\nYou will play a key role in migrating from an IaaS Big Data platform to Azure-native Databricks, optimizing data workflows and improving data quality. Join us to contribute to innovative solutions that boost client services and regulatory compliance.\nResponsibilities\nMigrate and optimize over 500 data jobs using Azure Databricks optimization techniques\nManage and process 12 TB of data efficiently across platforms\nTune machine learning models for Azure environments using Java Spark and Delta tables\nUpdate and maintain libraries to address security vulnerabilities\nDevelop and maintain ETL/ELT pipelines using PySpark and related technologies\nCollaborate with cross-functional teams to integrate GenAI capabilities into data workflows\nMonitor data quality and implement improvements to ensure accuracy and reliability\nAutomate deployment and operational tasks using Terraform and GitLab CI/CD\nSupport data governance initiatives to comply with regulatory standards\nTroubleshoot and resolve performance issues in data processing systems\nDocument system processes and provide technical guidance to junior engineers\nImplement best practices for code quality and data security\nParticipate in code reviews and knowledge sharing sessions\nOptimize costs associated with data storage and processing\nRequirements\nProficiency in Python and Spark with at least 3 years in data engineering roles\nStrong experience with Azure Databricks and PySpark \nProven expertise in designing and implementing ETL/ELT solutions\nExperience migrating big data platforms to Azure-native services\nProficiency with Delta tables for model tuning\nKnowledge of data governance and regulatory compliance frameworks\nFamiliarity with Docker, Kubernetes (AKS), and Terraform for infrastructure automation\nAbility to manage large data volumes with high efficiency\nExcellent problem-solving and analytical skills\nStrong communication and collaboration abilities\nEnglish proficiency at B2 level or higher\nWe offer\nWe gather like-minded people:\nTop tech minds driving innovation in AI, cloud and digital platform modernization\nSupportive team and agile, startup-like culture\nHybrid by design mode and opportunity to work remotely within Poland\nChance to work abroad for up to 60 days annually\nBusiness-driven relocation opportunities\n\nWe provide growth opportunities:\nCareer development programs\nThought leadership, mentoring, soft skills and well-being programs\nCertification (Anthropic, Gemini, GCP, Azure, AWS)\nEnglish classes\n\nWe cover it all:\nStable pay\nParticipation in the Employee Stock Purchase Plan with a 15% discount\nBenefits package (health insurance, multisport, shopping vouchers)\nReferral bonuses up to $2,000\nOffices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more\nCorporate, social and well-being events\n\nPlease, note:\nBenefits listed above are available to employees only\nWe are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually\nWe will reach out to selected candidates exclusively\n\nEPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.\nTech stack\nPySpark\nAzure Databricks\nPython\nApache Spark\nETL/ELT\nDelta Lake\ndata governance","description_format":"text","description_chars":3964,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Growth opportunities","Health insurance"],"hiring_locations":[{"name":"Poland","iso":"PL","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-10-08T23:50:21Z"}],"visa":[],"liveness":{"score":28,"band":"fade","label":"Fading","p_open":0.85,"p_active":0.741,"p_room":0.45,"age_days":23,"expected_fill_days":14,"reasons":["seen:23","velocity","win:tail","comp:brand"],"computed_at":"2026-10-31T05:58:55Z"},"pay":null,"html_url":"https://alion.io/job/epam-systems-poland-seniorlead-data-software-engineer-python-spark-azure","json_url":"https://alion.io/job/epam-systems-poland-seniorlead-data-software-engineer-python-spark-azure.json","meta":{"generated_at":"2026-11-01T02:55:04Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4460,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-02T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2612641},"rest":"https://alion.io/mcp/rest/get_company?id=2612641"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fepam-systems-poland-seniorlead-data-software-engineer-python-spark-azure"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fepam-systems-poland-seniorlead-data-software-engineer-python-spark-azure"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fepam-systems-poland-seniorlead-data-software-engineer-python-spark-azure"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/epam-systems-poland-seniorlead-data-software-engineer-python-spark-azure\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fepam-systems-poland-seniorlead-data-software-engineer-python-spark-azure"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":2123929}},{"tool":"search_jobs","arguments":{"company":"epam-poland.pl","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":2123929},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}