{"id":2304987,"url":"https://alion.io/job/photon-data-engineer-chennai-bengaluru","title":"Data Engineer - Chennai / Bengaluru","company":{"id":3186451,"name":"Photon","domain":"photon.com","url":"https://alion.io/company/photon-com","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Oracle","truth_index":{"grade":"D","score":45,"open_postings":598,"ghost_share":0.913,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-21T05:46:15Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India","Chennai, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":12500,"max_usd":25000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":71},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Airflow","optional":false},{"name":"Apache Kafka","optional":false},{"name":"CI/CD","optional":false},{"name":"Git","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Scrum","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"SSIS","optional":false},{"name":"Superset","optional":false},{"name":"Trino","optional":false},{"name":"Linux","optional":true},{"name":"OpenShift","optional":true}],"status":"live","first_seen_at":"2025-02-05T06:31:33Z","employer_posted_date":"2025-02-05","last_verified_at":"2026-10-21T23:29:48Z","board_verified":true,"closed_at":null,"days_open":623,"trust":{"level":"ghost","repost_count":0,"flags":["stale","company_stale"],"days_open":622},"description":"Data Engineer :\nJob Description :\nDevelop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python and PySpark to ensure the efficient and reliable delivery of data.\nDesign and implement custom connectors to facilitate the ingestion of diverse data sources into our platform, including structured and unstructured data from various document formats .\nCollaborate closely with cross-functional teams to gather requirements, understand data needs, and translate them into technical solutions.\nImplement DataOps principles and best practices to ensure robust data operations and efficient data delivery.\nDesign and implement data CI/CD pipelines to enable automated and efficient data integration, transformation, and deployment processes.\nMonitor and troubleshoot data pipelines, proactively identifying and resolving issues related to data ingestion, transformation, and loading.\nConduct data validation and testing to ensure the accuracy, consistency, and compliance of data.\nStay up-to-date with emerging technologies and best practices in data engineering.\nDocument data workflows, processes, and technical specifications to facilitate knowledge sharing and ensure data governance.\nResponsibilities:\nBachelor's degree in Computer Science, Engineering, or a related field\n5 + years experience in data engineering, ELT development, and data modeling.\nProficiency in using Apache Airflow and Spark for data transformation, data integration, and data management.\nExperience implementing workflow orchestration using tools like Apache Airflow, SSIS or similar platforms.\nDemonstrated experience in developing custom connectors for data ingestion from various sources.\nStrong understanding of SQL and database concepts, with the ability to write efficient queries and optimize performance.\nExperience implementing DataOps principles and practices, including data CI/CD pipelines.\nExcellent problem-solving and troubleshooting skills, with a strong attention to detail.\nEffective communication and collaboration abilities, with a proven track record of working in cross-functional teams.\nFamiliarity with data visualization tools Apache SuperSet and dashboard development.\nUnderstanding of distributed systems and working with large-scale datasets.\nFamiliarity with data governance frameworks and practices.\nKnowledge of data streaming and real-time data processing technologies (e.g., Apache Kafka).\nStrong understanding of software development principles and practices, including version control (e.g., Git) and code review processes.\nExperience with Agile development methodologies and working in cross-functional Agile teams.\nAbility to adapt quickly to changing priorities and work effectively in a fast-paced environment.\nExcellent analytical and problem-solving skills, with a keen attention to detail.\nStrong written and verbal communication skills, with the ability to effectively communicate complex technical concepts to both technical and non-technical stakeholders.\nRequired Skills -\nPython, Pyspark, Sql,Airflow, Trino, Hive, Snowflake, Agile Scrum\nGood to have-\nLinux,Openshift, Kubernentes, Superset","description_format":"text","description_chars":3153,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation"],"lifecycle":[{"event":"open","at":"2026-10-11T22:25:28Z"}],"visa":[],"liveness":{"score":4,"band":"cold","label":"Long shot","p_open":1,"p_active":0.149,"p_room":0.28,"age_days":622,"expected_fill_days":25,"reasons":["conf:3","stale_co","velocity","ghost","win:tail","crowd:brand"],"computed_at":"2026-10-21T05:46:15Z"},"pay":null,"html_url":"https://alion.io/job/photon-data-engineer-chennai-bengaluru","json_url":"https://alion.io/job/photon-data-engineer-chennai-bengaluru.json","meta":{"generated_at":"2026-10-22T00:35:33Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":877,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3186451},"rest":"https://alion.io/mcp/rest/get_company?id=3186451"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fphoton-data-engineer-chennai-bengaluru"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fphoton-data-engineer-chennai-bengaluru"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fphoton-data-engineer-chennai-bengaluru"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/photon-data-engineer-chennai-bengaluru\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fphoton-data-engineer-chennai-bengaluru"}]}