Sr Databricks Certified Data Engineer

Company Description

Derex Technologies Inc specializes in providing IT consulting, staffing solutions and software services. Globally headquartered in Harrison New Jersey since 1996 Derex delivers the highest quality technology professionals and an array of customized IT talent solutions designed to improve productivity and drive results to global clients throughout North America.

With over two decades of unparalleled experience, Derex provides supports to its clientele, across such industries as Systems Integration, Banking and Finance, Telecommunications, Pharmaceutical and Life Sciences, Energy, Healthcare, Technology, Transportation, and local and federal Government agencies.

Job Description

Position: Sr Databricks Certified Data Engineer

Location: Remote

Duration:Full Time

Job Description

• Sr Databricks Certified Data Engineer
• Must Have: Databricks Certified Data Engineer Professional Certification.
• Strongly preferred: Prior experience working in the Databricks Partner Program Databricks Partner Champion Certificate

• 10+ years of experience in data engineering, data platforms & analytics.
• Comfortable writing code in either Python or Scala.
• Extensive Working knowledge of Databricks is required.
• Experience with advanced Databricks concepts such as Genie Spaces, Agents, and Genie Code.
• Experience handling or leading large scale projects/customers.
• Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one.
• Deep experience with distributed computing with Apache Spark™ and knowledge of Apache Spark™ runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps Capable of design and deployment of highly performant end-to-end data architectures Experience with technical project delivery - managing scope and timelines Documentation and white-boarding skills Experience working with clients and managing conflicts Experience in building scalable streaming and batch solutions using cloud-native components

Here is some feedback I got for one of the consultant that took the interview:

1) Not strong in Data engineering and advanced concepts.

2) No Experience with advanced Databricks concepts such as Genie Spaces, Agents, and Genie Code.

3) No experience handling or leading large scale projects/customers.

Key Skills:

• They want someone with. Spark fundamentals - Spark architecture, DataFrames, partitions, shuffles, joins, and performance behavior.
• Delta Lake - ACID tables, schema evolution, merges, optimization, and reliable lakehouse storage patterns.
• ETL / pipeline design - batch and incremental pipelines, medallion-style thinking, and production pipeline design.
• Lakeflow /SDP / Jobs / workflows - orchestration, declarative pipelines, and workload deployment on Databricks.
• Ingestion patterns - landing data from databases, files, streams, and connectors into bronze/silver/gold pipelines.
• Performance tuning - cluster sizing, file layout, pipeline optimization, and Spark tuning best practices.
• Data modeling and warehousing - dimensional modeling, warehouse migration patterns, and serving BI/reporting use cases.
• CI/CD and deployment - asset bundles, deployment workflows

Regards,

Manoj Goud

Derex Technologies INC

Contact :973-834-5005 Ext 206

Additional Information

All your information will be kept confidential according to EEO guidelines.