Data Engineer (Lead / Architect)

Job Description

• Contribute using our standard tech stack, which is Python, Linux (via

  WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS

  cloud infrastructure.

• Participate in the design and implementation

  of advanced mechanisms for data ingestion and transformation, and mass

  parallel orchestration of network IO.

• Maintaining Infrastructure as

  Code and working with infrastructure teams to develop valuable new

  features for Client's Engineering Platform.

• Model data for a

  variety of purposes (operational and analytical) in collaboration with

  your team, other teams, and analytical data customers.

• Contribute

  to CLIENT's data platform in a way that promotes the development of a

  data mesh architecture for the entire enterprise.

• Work on indicators of data quality across our estate and improve the quality of our data along key metrics.
• Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications.
• Contribute to quality automation in our CI/CD process, Optimizing for bug-free release cadence and developer delight.
• Maintain good quality documentation (starting with your code) for technical and non-technical audiences.
• Participate in a pairing-first development team, learning skills from

  your colleagues and giving back to the community by imparting hard-won

  wisdom and skills learned from past experiences.

• Mentoring and coaching of less experienced team members through design and code reviews
• Participate in a safe and honest workplace culture by participating in

  open conversations about technical design, implementation and interrogating decisions when appropriate.

• Contribution to test automation through either enhancement to the platform or additional tests
• Assessment of application security to ensure it complies with CLIENT Application Security Standards
• Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards

Requirements

• Lead Level: (Exp Min 6+ Years)
• Architect: (Exp Min 9+ Years)
• Ability to articulate and advocate technology solutions effectively to

  technical and non-technical stakeholders while applying agile

  methodologies

• Proficiency in data interrogation and presentation of analysis results as part of team outputs
• Extensive experience as a senior software developer or engineer with a strong focus on Python development
• Strong expertise in SQL databases including PostgreSQL, SQL Server, and Databricks
• Comfortable working directly with databases without using Object Relational Mapping tools
• Experience working with data at scale and the associated technical challenges
• Proficiency with command line tools such as git, bash or zsh, and optionally vim
• An

  undergraduate degree or demonstrable knowledge in Computer Science,

  Software Engineering, Logic, Applied Statistics, Actuarial Science, Data

  Science, or related fields is preferred

Skills & Ability

• You are able to grasp, articulate, and

  advocate for technological solutions in both technical and

  non-technical settings and intelligently apply agile principles as you

  do so.

• You are able to interrogate and present data and data

  analysis and have a willingness to present results of analysis

  undertaken as a part of your team's work.

• You have demonstrable experience as a senior software developer or engineer with a focus on Python.
• SQL (Postgres, SQL Server, Databricks)
• Comfortable working without an ORM.
• Familiar with the challenges of working with data*at scale*.
• Prefer the command line use of git, bash/zsh, and (optionally) vim.
• An undergraduate degree or demonstrable knowledge and experience of

  concepts in Computer Science, Software Engineering, Logic, Applied

  Statistics, Actuarial Science, Data Science, or any related discipline

  would be advantageous.