{"id":697768,"url":"https://alion.io/job/mindera-senior-data-engineer-2","title":"Senior Data Engineer","company":{"id":54054,"name":"Mindera","domain":"mindera.com","url":"https://alion.io/company/mindera","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workable","truth_index":{"grade":"A","score":89,"open_postings":29,"ghost_share":0,"stale_share":0.241,"repost_share":0,"time_to_fill_p50_days":73,"computed_at":"2026-10-27T05:46:45Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Chennai, India","Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":18500,"max_usd":40000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":75},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Glue","optional":false},{"name":"AWS Lambda","optional":false},{"name":"ETL/ELT","optional":false},{"name":"IAM","optional":false},{"name":"Machine Learning","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Agile","optional":true},{"name":"CI/CD","optional":true},{"name":"Dimensional Modeling","optional":true},{"name":"Docker","optional":true},{"name":"Git","optional":true},{"name":"Scrum","optional":true}],"status":"live","first_seen_at":"2026-10-26T09:34:22Z","employer_posted_date":"2026-10-26","last_verified_at":"2026-10-27T03:33:45Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":1,"flags":[],"days_open":0},"description":"Senior Data Engineer - Mindera\nAt Mindera, we build technology products that create meaningful business impact. We work with modern cloud technologies and encourage engineering ownership, collaboration, continuous learning, and self-organization.\nOur Data Engineering teams build scalable, reliable, and high-performance data platforms that enable analytics, reporting, machine learning, and data-driven decision-making.\nRequirements\nWhat We’re Looking For\nKey Responsibilities\nDesign and develop scalable ETL/ELT data pipelines using AWS Glue, PySpark, Python, and SQL.\nDevelop and maintain distributed data processing applications using Apache Spark.\nBuild and orchestrate data workflows using Apache Airflow.\nDevelop AWS Glue jobs for batch and incremental data processing.\nWork with Amazon S3 and other AWS services to build cloud-based data solutions.\nPerform complex data transformations, cleansing, validation, and enrichment.\nWrite optimized and maintainable SQL queries for large datasets.\nOptimize Spark and PySpark jobs for performance, scalability, and resource utilization.\nImplement data partitioning, caching, broadcast joins, and other Spark optimization techniques.\nDesign reliable and fault-tolerant data pipelines with appropriate error handling and retry mechanisms.\nImplement data quality checks and monitoring across data pipelines.\nTroubleshoot production data pipeline failures and performance issues.\nWork with large-scale structured and semi-structured datasets.\nParticipate in technical design and architecture discussions.\nConduct code reviews and ensure adherence to engineering best practices.\nRequired Skills\nPython\nStrong programming experience in Python.\nExperience writing production-quality, modular, and reusable code.\nKnowledge of exception handling, logging, testing, and debugging.\nFamiliarity with Python data-processing libraries.\nSQL\nStrong SQL skills with experience handling large datasets.\nComplex joins and subqueries.\nCTEs and window functions.\nAggregations and analytical queries.\nQuery optimization and performance tuning.\nData deduplication and incremental processing logic.\nApache Spark / PySpark\nStrong hands-on experience with Spark and PySpark.\nUnderstanding of Spark architecture and execution model.\nDataFrames and Spark SQL.\nTransformations and actions.\nPartitioning and repartitioning.\nShuffle optimization.\nBroadcast joins.\nCaching and persistence.\nHandling data skew.\nSpark job troubleshooting and performance optimization.\nAWS Glue\nHands-on experience developing AWS Glue ETL jobs.\nExperience with Glue Data Catalog and Crawlers.\nGlue Spark jobs.\nDynamicFrames and DataFrames.\nJob bookmarks and incremental processing.\nPartitioned data processing.\nSchema evolution and data quality.\nIntegration with Amazon S3 and other AWS services.\nApache Airflow\nExperience designing and developing Airflow DAGs.\nTask dependencies and scheduling.\nOperators and sensors.\nRetries and failure handling.\nBackfills and catchup.\nXCom and task communication.\nMonitoring and troubleshooting DAGs.\nDesigning idempotent and reusable workflows.\nAWS\nStrong experience with AWS data services.\nAmazon S3.\nAWS Glue.\nIAM.\nCloudWatch.\nFamiliarity with services such as Lambda, Athena, Redshift, or EMR is an advantage.\nPreferred Skills\nExperience with modern data lake / data warehouse architectures.\nExperience with Parquet and other columnar storage formats.\nKnowledge of data modeling and dimensional modeling.\nExperience with CI/CD and Git-based development workflows.\nFamiliarity with Docker and infrastructure-as-code concepts.\nExperience with Agile/Scrum methodologies.\nExperience working with high-volume and large-scale datasets.\nUnderstanding of data governance, security, and data quality practices.\nSenior-Level Expectations\nIndependently own data engineering projects from requirements through production.\nDesign scalable and maintainable data architectures.\nMake appropriate technology and architecture decisions.\nIdentify and resolve performance bottlenecks.\nTroubleshoot complex production issues.\nEstablish coding, testing, and data quality standards.\nMentor other engineers and contribute to technical decision-making.\nCommunicate effectively with both technical and non-technical stakeholders.\nBenefits\nWe offer\nFlexible working hours (self-managed)\nAnnual bonus, subject to company performance\nAccess to Udemy online training and opportunities to learn and grow within the role\nAt Mindera we use technology to build products we are proud of, with people we love.\nSoftware Engineering Applications, including Web and Mobile, are at the core of what we do at Mindera.\nWe partner with our clients, to understand their products and deliver high-performance, resilient and scalable software systems that create an impact on their users and businesses across the world.\nYou get to work with a bunch of great people, and the whole team owns the project together.\nOur culture reflects our lean and self-organisation attitude.\nWe encourage our colleagues to take risks, make decisions, work in a collaborative way and talk to everyone to enhance communication. We are proud of our work and we love to learn all and everything while navigating through an Agile, Lean and collaborative environment.\nOur offices are located: Porto, Portugal | Aveiro, Portugal | Coimbra, Portugal | Leicester, UK | San Diego, USA | Chennai, India | Bengaluru, India","description_format":"text","description_chars":5386,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Continuous learning","Flexible schedule"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Outsourcing & Dedicated Teams","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-09-11T02:14:20Z"},{"event":"close","at":"2026-09-14T11:59:26Z"},{"event":"reopen","at":"2026-10-26T05:25:15Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":0,"expected_fill_days":73,"reasons":["conf:2","velocity","win:early"],"computed_at":"2026-10-27T05:46:45Z"},"pay":null,"html_url":"https://alion.io/job/mindera-senior-data-engineer-2","json_url":"https://alion.io/job/mindera-senior-data-engineer-2.json","meta":{"generated_at":"2026-10-27T07:13:03Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":7649,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-28T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":54054},"rest":"https://alion.io/mcp/rest/get_company?id=54054"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmindera-senior-data-engineer-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmindera-senior-data-engineer-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmindera-senior-data-engineer-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/mindera-senior-data-engineer-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmindera-senior-data-engineer-2"}]}