{"id":2390886,"url":"https://alion.io/job/koantek-databricks-platform-architect","title":"Databricks Platform Architect","company":{"id":3800091,"name":"Koantek","domain":"koantek.com","url":"https://alion.io/company/koantek","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Zoho Recruit","truth_index":{"grade":"B","score":70,"open_postings":22,"ghost_share":0.5,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-13T05:45:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"staff","employment_type":"contractor","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Chandler, United States"],"countries":["US"],"hiring_countries":["US"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":157000,"max_usd":270000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":93},"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Claude","optional":false},{"name":"Databricks","optional":false},{"name":"dbt","optional":false},{"name":"Delta Lake","optional":false},{"name":"FinOps","optional":false},{"name":"Git","optional":false},{"name":"MS SQL","optional":false},{"name":"pySpark","optional":false},{"name":"Pytest","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Terraform","optional":false},{"name":"Python","optional":true}],"status":"live","first_seen_at":"2026-05-15T00:00:00Z","employer_posted_date":"2026-05-15","last_verified_at":"2026-10-13T07:14:10Z","board_verified":true,"closed_at":null,"days_open":151,"trust":{"level":"ghost","repost_count":0,"flags":["stale","company_stale"],"days_open":151},"description":"Job Title: Lead Data Platform Architect / Data bricks Migration Lead Location: Remote Position Type: ContractJob Overview We are seeking an accomplished, technology-driven Lead Data Platform Architect / Migration Specialist to spearhead the modernization of our core enterprise financial and tax allocation engines. In this role, you will lead the architectural design, definition of migration strategies, and hands-on implementation to transition large-scale legacy relational database systems (SQL Server/T-SQL) into a modern, cloud-native Databricks Lakehouse platform. The ideal candidate will have extensive experience in high-throughput distributed systems, Databricks compute optimization, performance tuning, and complex pipeline orchestration.\nKey\nResponsibilities Architecture & Strategy: Validate, refine, and own the target architecture on Databricks. Define robust migration strategies and production-ready reference patterns to convert 150+ complex stored procedures into PySpark and Structured/Declarative Pipelines (SDP). Pipeline Engineering: Design distributed processing frameworks, control flows, and configuration-driven parameter handling for both full and incremental recalculation modes. Performance Optimization: Address performance deltas between small and large workloads. Architect and implement acceleration techniques such as caching, partition pruning, cluster sizing, and offline/pre-calculation strategies to maintain sub-30-second user-facing reporting SLAs. Orchestration & Observability: Design and deploy enterprise-level pipeline orchestration using tools like Apache Airflow or Databricks Workflows. Integrate robust logging, error handling, and observability patterns into existing enterprise monitoring frameworks. Governance & Security: Implement data governance models, data lineage, and schema evolution utilizing tools like Unity Catalog. AI-Assisted Delivery & Code Quality: Establish best practices for AI-assisted code generation (e.g., using Claude or advanced LLMs), providing code-review patterns and refactoring frameworks to ensure maintainable and performant output. Team Enablement: Lead code walkthroughs, design reviews, and pair-programming sessions with the development team to accelerate knowledge transfer and technical excellence. Required Technical Skills &\nQualifications Core Big Data Platform: Deep expert-level knowledge of Databricks (Lakehouse architecture, Delta Lake, Unity Catalog) and Apache Spark / PySpark. Legacy Database Expertise: Strong background in relational databases, with advanced proficiency in SQL Server, T-SQL, and Stored Procedures. Ability to reverse-engineer and refactor legacy database logic into distributed paradigms. Orchestration Tools: Hands-on experience with Apache Airflow or similar modern workflow orchestrators. Performance Tuning: Proven track record in cost optimization (FinOps), cluster tuning, autoscaling configurations, and handling skewed data profiles. CI/CD & DevOps: Experience with Infrastructure as Code (Terraform), data build tool (dbt), testing frameworks (PyTest), and automated Git-based workflows. Experience Level: 10+ years of experience in Data Engineering/Architecture, with at least 3+ years specifically leading large-scale cloud data migrations.\nEducation: Bachelor’s or Master's degree in Computer Science, Engineering, or a related technical field. Preferred\nCertifications Databricks Certified Data Engineer Associate / Professional Databricks Certified Solutions Architect AWS Certified Database Specialist or equivalent Cloud Certifications","description_format":"text","description_chars":3576,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Analytics & BI Consulting","Data Engineering & Migration Services"],"lifecycle":[{"event":"open","at":"2026-10-12T21:11:42Z"}],"visa":[],"liveness":{"score":7,"band":"cold","label":"Long shot","p_open":1,"p_active":0.24,"p_room":0.28,"age_days":151,"expected_fill_days":7,"reasons":["conf:1","stale_co","ghost","win:tail","crowd:"],"computed_at":"2026-10-13T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/koantek-databricks-platform-architect","json_url":"https://alion.io/job/koantek-databricks-platform-architect.json","meta":{"generated_at":"2026-10-13T07:39:05Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":11293,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-14T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3800091},"rest":"https://alion.io/mcp/rest/get_company?id=3800091"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fkoantek-databricks-platform-architect"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fkoantek-databricks-platform-architect"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fkoantek-databricks-platform-architect"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/koantek-databricks-platform-architect\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fkoantek-databricks-platform-architect"}]}