Senior Data Engineer

Responsibilities:

• Maintain our data platform with timely and quality data
• Plan and execute data platform expansion to support the company’s growth and analytic needs. Define and extend our internal standards for style, maintenance, and best practices for a high-scale data platform
• Build and maintain data pipelines from internal databases and SaaS applications
• Write maintainable, performant code and create and maintain systems documentation
• Desire to continually keep up with advancements in data engineering practices.
• Solve technical problems of the highest scope and complexity
• Implement the DataOps philosophy in everything you do
• Design and develop code to extend the enterprise data model
• Build trust in all interactions and with trusted data development
• Collaborate with Analytics Engineers and Data Analysts to drive efficiencies for their work
• Collaborate with other functions to ensure data needs are addressed

Requirements:

• Python/Scala/Java
• Data Processing(Apache Spark/Apache Flink/Apache Beam, etc.) and Pipeline orchestration (Apache Airflow, Apache NiFi end, etc)
• Message Brokers and distributed streaming platform
• DB (SQL/NoSQL/Open formats)
• Dev Tools (Git/Docker/Kubernetes/Jira/etc)
• Infrastructure/Cloud (AWS/GCP/Azure/etc).
• Data Privacy/Security
• Algorithms/Data Structure/Data Modeling
• Data Development Principles/Process
• Writing Requirements/Documentation
• CI/CD/CD
• Practical familiarity with DataOps, Data Lake, Data Mart, and DataMesh/Data Hub principles
• Nice to have: AWS Redshift, AWS Glue, S3, EMR, DMS, QuickSight, MWAA, etc