Senior Data Architect - Databricks

Responsibilities:

• Handle a variety of impactful customer technical projects that may include designing and building reference architectures, creating how-tos, and professionalizing customer use cases
• Work with engagement managers to scope a variety of professional services and work with input from the customer
• Guide strategic customers as they implement transformational big data projects and 3rd party migrations, including end-to-end design, build, and deployment of industry-leading big data and AI applications
• Consult on architecture and design; bootstrap or implement customer projects that lead to a customer's successful understanding, evaluation, and adoption of Databricks.
• Provide an escalated level of support for customer operational issues
• Collaborate with the project manager, architect, and customer teams to ensure the technical components of the engagement are delivered to meet customer needs
• Mentor and guide junior data engineers and team members.

Requirements:

• 6+ years of experience in data engineering, data platforms, and analytics on the Databricks platform.
• Experience with large-scale data migrations to/from Databricks.
• Consulting/customer-facing experience, working with external clients across a variety of industry markets.
• Comfortable writing code in either Python or Scala.
• Working knowledge of two or more common cloud ecosystems (AWS, Azure, GCP) with expertise in at least one.
• Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals.
• Familiarity with CI/CD for production deployments.
• Working knowledge of MLOps.
• Design and deployment of performant end-to-end data architectures.
• Experience with technical project deliverymanaging scope and timelines.
• Documentation and white-boarding skills.
• Experience working with clients and managing conflicts.
• Deployment and integration of Databricks-based solutions to complete customer projects.
• Must have Databricks certifications.
• Strong communication and collaboration skills.
• Excellent problem-solving skills.
• Ability to travel up to 30% when needed.

Skills

Apache Spark, Databricks, Python, Scala, Spark