Forward Deployed Engineer with Databricks & Claude

Role Summary:

We are looking for a Forward Deployed Engineer with strong hands-on experience in Databricks and Generative AI/Claude to work closely with clients, business stakeholders, and internal engineering teams. The ideal candidate will combine strong Data Engineering, Software Engineering, Databricks, and Generative AI skills with the ability to understand business problems and rapidly build, deploy, and optimize production-ready solutions. This is a client-facing, hands-on engineering role where you will work from problem discovery and solution design through POC development, production deployment, and ongoing optimization.

Key Responsibilities:

Forward Deployed Engineering

• Work directly with clients and stakeholders to understand business and technical requirements.
• Translate business problems into scalable data, AI, and software solutions.
• Design and develop POCs and rapidly validate technical solutions.
• Convert successful POCs into reliable, production-ready applications.
• Work closely with client engineering and data teams during implementation and deployment.
• Troubleshoot production issues and continuously optimize deployed solutions.
• Act as a technical bridge between clients, delivery teams, data engineers, AI engineers, and architects.

Databricks & Data Engineering

• Design and develop scalable data solutions using Databricks, PySpark, Python, and SQL.
• Build and optimize data ingestion, transformation, and ETL/ELT pipelines.
• Work with Databricks Lakehouse, Delta Lake, and Unity Catalog.
• Develop Databricks Workflows and production data pipelines.
• Implement data processing solutions for structured and semi-structured datasets.
• Optimize Databricks workloads for performance, scalability, reliability, and cost.
• Integrate Databricks with databases, APIs, cloud platforms, and enterprise applications.

Generative AI & Claude

• Build enterprise AI solutions using Claude and other Large Language Models (LLMs).
• Integrate Claude APIs into applications and business workflows.
• Develop RAG (Retrieval-Augmented Generation) solutions using enterprise data.
• Work with embeddings, vector search, semantic search, and knowledge retrieval.
• Develop AI-powered applications for summarization, classification, information extraction, question answering, and document processing.
• Implement prompt engineering, structured outputs, tool/function calling, and context management.
• Develop and integrate AI agents and multi-step AI workflows where applicable.
• Evaluate LLM responses for accuracy, relevance, groundedness, latency, and cost.
• Implement appropriate AI guardrails, security, and data privacy controls.

Production & Deployment

• Deploy AI and data solutions into production environments.
• Work with APIs, microservices, Git, CI/CD, containers, and cloud platforms.
• Monitor application and pipeline performance and troubleshoot issues.
• Collaborate with Data Scientists and ML Engineers to productionize AI/ML models.
• Ensure solutions meet security, scalability, reliability, and maintainability requirements.

Required Skills & Experience

• 4+ years of experience in Data Engineering, Software Engineering, AI/ML Engineering, or a related field.
• Strong hands-on experience with Databricks.
• Strong proficiency in Python, PySpark, and SQL.
• Experience with Delta Lake and Lakehouse Architecture.
• Experience working with Generative AI / LLMs.
• Hands-on experience with Claude / Anthropic APIs is preferred.
• Experience with RAG, embeddings, vector databases, and semantic search.
• Strong understanding of REST APIs and enterprise integrations.
• Experience developing production-grade applications and data pipelines.
• Strong problem-solving and troubleshooting capabilities.
• Excellent communication and client-facing skills.

Preferred Skills

• Experience with Claude Code / Anthropic ecosystem.
• Experience with OpenAI, Azure OpenAI, AWS Bedrock, or other LLM platforms.
• Experience with LangChain, LangGraph, LlamaIndex, or equivalent frameworks.
• Experience with Databricks Unity Catalog, Workflows, and MLflow.
• Experience with AWS, Azure, or GCP.
• Experience with Docker, Kubernetes, and CI/CD.
• Exposure to AI agents and agentic workflows.
• Knowledge of AI evaluation, guardrails, security, and responsible AI.
• Experience working in consulting, client delivery, or customer-facing engineering environments.

Key Competencies

• Strong customer-facing and stakeholder management skills.
• Ability to understand ambiguous business problems and translate them into technical solutions.
• Strong ownership and execution mindset.
• Ability to rapidly prototype, iterate, and productionize solutions.
• Strong analytical and troubleshooting skills.
• Comfortable working in fast-paced and dynamic client environments.
• Excellent written and verbal communication.
• Ability to work independently as well as collaboratively with distributed teams.

Skills

Python, SQL, databricks