Software Development Engineer II - Data

We’re looking for a seasoned Software Development Engineer (SDE II) - Data Engineering with deep expertise in building large-scale data ingestion, processing, and analytics platforms. In this role, you’ll collaborate closely with Product, Design, and cross-functional stakeholders to design and implement high-throughput data pipelines and lakehouse solutions that power analytics, AI, and real-time decision-making for predicting and preventing cyber breaches.

What You’ll Do:

• Design and Develop: Architect and implement high-scale data pipelines leveraging Apache Spark, Flink, and Airflow to process streaming and batch data efficiently.
• Data Lakehouse and Storage Optimization: Build and maintain data lakes and ingestion frameworks using Snowflake, Apache Iceberg, and Parquet, ensuring scalability, cost efficiency, and optimal query performance.
• Data Modeling and System Design: Design robust, maintainable data models to handle structured and semi-structured datasets for analytical and operational use cases.
• Real-time and Batch Processing: Develop low-latency pipelines using Kafka and Spark Structured Streaming, supporting billions of events per day.
• Workflow Orchestration: Automate and orchestrate end-to-end ELT processes with Airflow, ensuring reliability, observability, and recovery from failures.
• Cloud Infrastructure: Build scalable, secure, and cost-effective data solutions leveraging AWS native services (S3, Lambda, ECS, etc.).
• Monitoring and Optimization: Implement strong observability, data quality checks, and performance tuning to maintain high data reliability and pipeline efficiency.

What We’re Looking For:

• Bachelor’s or Master's degree in Computer Science, Engineering, or a related field
• 3+ years of experience in data engineering with a proven track record of designing large-scale, distributed data systems.
• Strong expertise in Snowflake and other distributed analytical data stores.
• Hands-on experience with Apache Spark, Flink, Airflow, and modern data lakehouse formats (Iceberg, Parquet).
• Deep understanding of data modeling, schema design, query optimization, and partitioning strategies at scale.
• Proficiency in Python, SQL, Scala, Go/Nodejs with strong debugging and performance-tuning skills.
• Experience in streaming architectures, CDC pipelines, and data observability frameworks.
• Proficient in deploying containerized applications (Docker, Kubernetes, ECS).
• Familiarity with using AI Coding assistants like Cursor, Claude Code, or GitHub Copilot

Preferred Qualifications:

• Exposure to CI/CD pipelines, automated testing, and infrastructure-as-code for data workflows.
• Familiarity with streaming platforms (Kafka, Kinesis, Pulsar) and real-time analytics engines (Druid, Pinot, Rockset).
• Understanding of data governance, lineage tracking, and compliance requirements in a multi-tenant SaaS platform.