Senior Data Engineer

• Assess and re-engineer ingestion pipelines from operational and source-system data
• Design and implement robust data mappings, lineage, dependencies and controlled migration or cutover approaches
• Develop and maintain detailed data reconciliation processes
• Refactor and enhance PySpark-based data processing and hierarchy/roll-up capabilities
• Introduce testing, versioning, metadata, observability and deployment controls
• Collaborate with business and technical SMEs to improve data coverage
• Assess GCP data pipeline options and contribute to implementation
• Assess opportunities to source mappings and reference data directly from systems such as Microsoft Dynamics 365
• Support reporting hierarchy and filtering logic changes, including impact assessment, testing and reconciliation
• Build and maintain automated data-quality checks, monitoring and operational controls
• Support adoption of centralised reporting and data capabilities
• Produce technical documentation, decision records, reconciliation evidence, variance explanations and handover materials
• Deliver through a prioritised backlog and short iterative sprints with engineers and stakeholders

• 5+ years of professional experience in Data Engineering or closely related data roles, with significant hands-on experience building and supporting production data solutions
• Strong hands-on experience with PySpark, SQL and Databricks, including the development and support of production-grade data pipelines
• Hands-on experience with GCP data technologies and data pipelines
• Experience with data pipeline testing, optimisation and operational support
• Strong experience with dimensional, master/reference or hierarchical data modelling
• Experience implementing complex entity matching, data mappings and roll-up logic with appropriate traceability
• Practical experience implementing automated data-quality checks and reconciliation processes
• Experience with observability, audit history, version control and CI/CD within data engineering solutions
• Experience assessing source-system data and documenting data contracts, mappings, lineage and dependencies
• Experience managing controlled data pipeline or platform transitions
• Ability to investigate data issues and clearly explain variances in values, record counts, mappings and hierarchy outcomes
• Ability to work directly with platform teams, analysts and business SMEs, translating ambiguous reporting and mapping needs into tested engineering solutions
• Strong communication skills with the ability to produce concise technical documentation, decision records, variance explanations and handover materials
• Self-motivated and comfortable working within a collaborative, iterative delivery environment
• Preferred: Experience working within Insurance, London Market, Reinsurance, broker, distribution or policy-level data environments
• Preferred: Experience integrating with Microsoft Dynamics 365 or comparable operational source systems
• Preferred: Experience with Power BI, including semantic models and reporting dependencies
• Preferred: Experience with Slowly Changing Dimensions (SCDs) and historical hierarchy management
• Preferred: Experience supporting complex master/reference data, entity matching and hierarchy solutions
• Preferred: Experience investigating historical data and reporting discrepancies
• Preferred: Experience supporting controlled data migrations, source-system transitions or platform changes
• Preferred: Experience working within distributed Agile teams and collaborating with UK-based business and technical stakeholders