{"id":3776559,"url":"https://alion.io/job/mindbrain-senior-data-engineer-with-java-python-5-years","title":"Senior Data Engineer with Java /Python (5+ Years)","company":{"id":4442882,"name":"MindBrain","domain":"mindbrain.in","url":"https://alion.io/company/mindbrain","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["India"],"countries":["IN"],"hiring_countries":["IN"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"Datadog","optional":false},{"name":"dbt","optional":false},{"name":"Docker","optional":false},{"name":"Flink","optional":false},{"name":"Java","optional":false},{"name":"Kubernetes","optional":false},{"name":"Microsoft Fabric","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-11-04T17:26:59Z","employer_posted_date":null,"last_verified_at":"2026-11-04T17:26:59Z","board_verified":false,"closed_at":null,"days_open":2,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":2},"description":"Job Description – Data Engineer\nPosition: Data Engineer\nExperience: 5–7 Years\nRelevant Experience: 5+ Years\nLocation: Remote\nEmployment Type: Full-Time\nJob Summary\nWe are looking for an experienced Data Engineer with strong expertise in Python, SQL, Java or Scala, Apache Spark, and Microsoft Fabric. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data pipelines and data engineering solutions.\nThe candidate should have hands-on experience working with batch and real-time data processing, modern data frameworks, data modeling, and Microsoft Fabric Notebooks to build and operationalize efficient data workflows.\nKey Responsibilities\nDesign, develop, and maintain scalable and reliable data pipelines using Python, Java, or Scala.\nBuild and optimize data processing solutions using Apache Spark and modern data frameworks.\nDevelop batch and streaming data pipelines using technologies such as Kafka, Flink, Spark, and dbt.\nWork with SQL and NoSQL databases, implementing efficient data models and storage solutions.\nOptimize SQL queries, data pipelines, and storage systems for improved performance and cost efficiency.\nUtilize Microsoft Fabric Notebooks to develop, debug, test, and operationalize data engineering workflows.\nImplement data integration, transformation, and processing solutions to support business requirements.\nLeverage Docker and Kubernetes for containerization and orchestration of data engineering workloads.\nImplement monitoring, logging, and observability practices using tools such as Datadog.\nCollaborate with cross-functional teams to ensure data quality, reliability, scalability, and performance.\nRequired Skills & Qualifications\n5–7 years of overall experience in Data Engineering, with at least 5 years of relevant experience.\nStrong programming proficiency in Python, Java, or Scala.\nHands-on experience with Apache Spark and distributed data processing.\nPractical experience with Microsoft Fabric Notebooks for data engineering workflows.\nStrong SQL skills, including query development and performance optimization.\nExperience with modern data frameworks such as Kafka, Flink, dbt, or equivalent technologies.\nGood understanding of SQL and NoSQL databases and data modeling principles.\nExperience developing both batch and/or streaming data processing solutions.\nHands-on experience with Docker and Kubernetes.\nKnowledge of observability and monitoring tools, preferably Datadog.\nStrong analytical, problem-solving, and troubleshooting skills.\nMandatory Technical Skills\nPython | SQL | Java/Scala | Apache Spark | Microsoft Fabric\nGood to Have\nExperience with Kafka, Flink, and dbt.\nExposure to cloud-based data engineering environments.\nExperience with Datadog or similar monitoring and observability platforms.\nKnowledge of data pipeline orchestration and workflow automation.\nWork Details\nWork Mode: 100% Remote\nExperience: 5–7 Years\nRelevant Experience: Minimum 5 Years","description_format":"text","description_chars":2965,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Health Care","Psychotherapy & Counseling","Psychiatric Care"],"lifecycle":[{"event":"open","at":"2026-11-04T23:09:45Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":29,"reasons":["seen:1","velocity","win:early"],"computed_at":"2026-11-06T05:58:05Z"},"pay":null,"html_url":"https://alion.io/job/mindbrain-senior-data-engineer-with-java-python-5-years","json_url":"https://alion.io/job/mindbrain-senior-data-engineer-with-java-python-5-years.json","meta":{"generated_at":"2026-11-07T02:15:47Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":5335,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4442882},"rest":"https://alion.io/mcp/rest/get_company?id=4442882"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmindbrain-senior-data-engineer-with-java-python-5-years"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmindbrain-senior-data-engineer-with-java-python-5-years"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmindbrain-senior-data-engineer-with-java-python-5-years"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/mindbrain-senior-data-engineer-with-java-python-5-years\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmindbrain-senior-data-engineer-with-java-python-5-years"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3776559}},{"tool":"search_jobs","arguments":{"company":"mindbrain.in","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3776559},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}