Senior Data Engineer

We are seeking a Senior Data Engineer to operate & design, manage & maintain, and optimize our data platform. In this role, you will bridge the gap between traditional big data engineering and modern Generative AI infrastructure. You will be responsible for building robust data pipelines on Databricks, engineering high-quality datasets for AI models, and implementing retrieval architectures.

Key Responsibilities

• Data Architecture: Design scalable distributed data processing systems.
• Pipeline Development: Build ETL/ELT pipelines for complex datasets.
• AI Integration: Implement and fine-tune generative AI capabilities.
• App Engineering: Develop backend services to expose data products.
• System Optimization: Improve code performance, data quality, and storage efficiency.
• Mentorship: Guide junior engineers and advocate for technical best practices.

Required Technical Skills

Primary Expertise (Expert Level)

• Data Engineering: Apache Spark, Hadoop, Kafka, and Databricks/Delta Lake.
• Python Ecosystem: PySpark, Pandas, NumPy, and performance tuning.
• Cloud Platforms: Advanced deployment experience within AWS or Azure.
• Data Modeling: Expert knowledge of relational and non-relational databases.

Secondary Expertise (Intermediate Level)

• GenAI / LLMs: Experience with LangChain, LlamaIndex, and vector databases (e.g., Pinecone, Milvus).
• Application Engineering: Building APIs using FastAPI, Flask, or Node.js.
• DevOps / MLOps: Containerization via Docker, Kubernetes, and CI/CD deployment workflows.

Qualifications

• Education: Bachelor's degree in Computer Science or a related field.
• Experience: 7+ years of professional software engineering experience.
• Portfolio: Proven track record of shipping production-grade data pipelines.

About Adobe

Adobe empowers everyone to create through innovative platforms and tools that unleash creativity, productivity and personalized customer experiences. Adobe’s industry-leading offerings including Adobe Acrobat Studio, Adobe Express, Adobe Firefly, Creative Cloud, Adobe Experience Platform, Adobe Experience Manager, and GenStudio enable people and businesses to turn ideas into impact, powered by AI and driven by human ingenuity.

Our 30,000+ employees worldwide are creating the future and raising the bar as we drive the next decade of growth. We’re on a mission to hire the very best and believe in creating a company culture where all employees are empowered to make an impact. At Adobe, we believe that great ideas can come from anywhere in the organization. The next big idea could be yours.

Let’s Adobe together

At Adobe, we believe in creating a company culture where all employees are empowered to make an impact. Learn more about Adobe life, including our values and culture, focus on people, purpose and community, Adobe for All, comprehensive benefits programs, the stories we tell, the customers we serve, and how you can help us advance our mission of empowering everyone to create.

Adobe is proud to be an Equal Employment Opportunity employer. We do not discriminate based on gender, race or color, ethnicity or national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, or any other protected characteristic. Learn more.

Adobe aims to make our Careers website and recruiting process accessible to any and all users. If you have a disability or special need that requires accommodation to navigate our website or complete the application process, email [email protected] or call +1 408-536-3015.

AI Use Guidelines for Interviews:

Our interviews are designed to reflect your own skills and thinking. The use of AI or recording tools during live interviews is not permitted unless explicitly invited by the interviewer or approved in advance as part of a reasonable accommodation. If these tools are used inappropriately or in a way that misrepresents your work, your application may not move forward in the process.

At Adobe, we empower employees to innovate with AI - and we look for candidates eager to do the same. As part of the hiring experience, we provide clear guidance on where AI is encouraged during the process and where it’s restricted during live interviews. See how we think about AI in the hiring experience.