Data Engineer

We are seeking a Data Engineer to support delivery of key data components within our evolving data ecosystem . This role will work closely with architects, analysts, engineers , and wider delivery teams to implement pipeline builds, data models, and integration patterns aligned to the organisation’s new Target Operating Model for Data .

You will contribute to the development of scalable data flows , optimise ingestion & transformation activities, and ensure solutions meet technical standards, performance expectations, and security controls . This is a hands-on engineering role ideal for someone who enjoys building, shaping, and improving modern data platforms .

Who We’re Looking For

A highly capable Data Engineer with hands-on experience building ETL pipelines , implementing Data Lake architectures , and working across Azure platform services . You will need strong problem-solving ability , a solid engineering mindset , and the ability to work collaboratively within a multi-disciplinary team .

If you’re confident writing production-grade code , applying design patterns , and building scalable, high-performing data products , this role offers technical autonomy, ownership, and challenge .

Key Responsibilities

• Build and maintain ETL pipelines and code in cloud data platforms.
• Support ingestion activity and onboarding of new data sources.
• Work with the Databricks Data Intelligence Platform and

• develop
• Data Engineering workloads.

• Construct raw, refined and curated data layers; catalogue assets appropriately.
• Validate solutions against functional and non-functional requirements.
• Deliver datasets , transformations and performance-optimised data products.
• Improve processes, engineering patterns, and reusable tooling.
• Monitor and measure pipeline performance; support incident resolution.
• Ensure documentation meets acceptance standards and is approved centrally.
• Actively engage in Agile ceremonies and governance forums.

Mandatory Requirements

• Strong experience with Python, PySpark & SQL for data engineering .
• Hands-on experience with Azure Databricks .
• Strong knowledge of Azure Data Factory & Fabric .
• Experience with Microsoft Purview .
• Familiarity with Event-driven data ingestion (Event Grid / Pub-Sub) .
• Understanding of SOLID principles , Async programming, Mediator/Factory patterns.
• Experience delivering unit + integration testing in Databricks .
• Knowledge of Secure ETL design with Entra IMID/SCIM integration.
• Understanding of Azure best practice, APIM, and platform governance .
• Ability to build and serve Power BI models via Databricks data sources .

Desirable

• Prior experience working within UK Public Sector environments .

Soft Skills

• Strong stakeholder communication and cross-team collaboration.
• Analytical and solution-focused mindset.
• Able to work independently, take ownership and drive progress.
• Commitment to clean, scalable, well-documented engineering.
• Adaptable, proactive, and comfortable working in dynamic delivery environments.

Contract Details

• Contract Type: Freelance / Contract - Outside IR35
• Location: Remote (UK-based candidates preferred)
• Start Date: ASAP
• Clearance: Candidates must be eligible to work with UK Government departments / BPSS.