Data Engineer (Lead / Architect)

Job Description Contribute using our standard tech stack, which is Python, Linux (via
WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS
cloud infrastructure. Participate in the design and implementation
of advanced mechanisms for data ingestion and transformation, and mass
parallel orchestration of network IO. Maintaining Infrastructure as
Code and working with infrastructure teams to develop valuable new
features for Client's Engineering Platform. Model data for a
variety of purposes (operational and analytical) in collaboration with
your team, other teams, and analytical data customers. Contribute
to CLIENT's data platform in a way that promotes the development of a
data mesh architecture for the entire enterprise. Work on indicators of data quality across our estate and improve the quality of our data along key metrics. Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications. Contribute to quality automation in our CI/CD process, Optimizing for bug-free release cadence and developer delight. Maintain good quality documentation (starting with your code) for technical and non-technical audiences. Participate in a pairing-first development team, learning skills from
your colleagues and giving back to the community by imparting hard-won
wisdom and skills learned from past experiences. Mentoring and coaching of less experienced team members through design and code reviews Participate in a safe and honest workplace culture by participating in
open conversations about technical design, implementation and interrogating decisions when appropriate. Contribution to test automation through either enhancement to the platform or additional tests Assessment of application security to ensure it complies with CLIENT Application Security Standards Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards Requirements Lead Level: (Exp Min 6+ Years) Architect: (Exp Min 9+ Years) Ability to articulate and advocate technology solutions effectively to
technical and non-technical stakeholders while applying agile
methodologies Proficiency in data interrogation and presentation of analysis results as part of team outputs Extensive experience as a senior software developer or engineer with a strong focus on Python development Strong expertise in SQL databases including PostgreSQL, SQL Server, and Databricks Comfortable working directly with databases without using Object Relational Mapping tools Experience working with data at scale and the associated technical challenges Proficiency with command line tools such as git, bash or zsh, and optionally vim An
undergraduate degree or demonstrable knowledge in Computer Science,
Software Engineering, Logic, Applied Statistics, Actuarial Science, Data
Science, or related fields is preferred Skills & Ability You are able to grasp, articulate, and
advocate for technological solutions in both technical and
non-technical settings and intelligently apply agile principles as you
do so. You are able to interrogate and present data and data
analysis and have a willingness to present results of analysis
undertaken as a part of your team's work. You have demonstrable experience as a senior software developer or engineer with a focus on Python. SQL (Postgres, SQL Server, Databricks) Comfortable working without an ORM. Familiar with the challenges of working with data*at scale*. Prefer the command line use of git, bash/zsh, and (optionally) vim. An undergraduate degree or demonstrable knowledge and experience of
concepts in Computer Science, Software Engineering, Logic, Applied
Statistics, Actuarial Science, Data Science, or any related discipline
would be advantageous.