Senior Analytics Engineer

• Build and maintain AWS infrastructure for the analytics platform, including storage, compute, networking, data catalogue, secrets and access management
• Manage infrastructure as code in Terraform
• Administer the ClickHouse Cloud warehouse, including configuration, users, roles and performance/capacity monitoring
• Host and maintain self-hosted Apache Superset and Apache Airflow tools
• Contribute to analytics platform architecture decisions
• Build and maintain ingestion pipelines from application databases, APIs and operational systems using AWS Glue, Airbyte and AWS Lambda
• Implement efficient and recoverable incremental loading patterns
• Develop and optimise Spark jobs processing billions of device events
• Model programme context including schools, terms, device allocations and enrolment
• Design and maintain Apache Iceberg tables for performance and cost
• Validate data at ingestion and investigate quality issues
• Build and maintain Airflow workflows with dependencies, retries and alerting
• Develop dbt staging and core models in ClickHouse
• Implement incremental models balancing freshness, correctness and cost
• Maintain dbt tests and model contracts
• Document core models, lineage and approximations
• Collaborate with the Senior BI & Data Analyst and Data Scientist
• Build CI/CD pipelines in GitHub Actions
• Implement monitoring and alerting across pipelines, warehouse and hosted tools
• Respond to incidents, perform root cause analysis and support data quality processes
• Maintain access controls and security practices
• Monitor AWS and ClickHouse costs and optimise workloads
• Maintain and test backup and recovery procedures
• Participate in code reviews and write runbooks and technical documentation

• Bachelor’s degree in Computer Science, Software Engineering or a related field, or equivalent practical experience; demonstrated experience is weighted equally to formal credentials
• At least 7 years in data engineering, software engineering or platform engineering roles
• At least 3 years building and running distributed data processing (Spark or equivalent) in production
• Track record of responsibility for a production data platform or major parts of one, including reliability, performance and day-to-day operation
• Experience with datasets in the hundreds of millions to billions of records
• Experience making trade-offs between compute, storage and performance within real budget constraints
• Experience managing cloud infrastructure for data workloads
• Experience working in small teams with limited supervision
• Experience working with remote teams across multiple countries and time zones
• Strong Python with testing and modular design
• Advanced SQL for analytical databases
• Spark (PySpark), including performance tuning for partitioning, shuffles and data skew
• Understanding of data lake design, Parquet, partitioning strategy, and lake-versus-warehouse processing
• Hands-on AWS experience, including S3, Glue or EMR, IAM and EC2
• Infrastructure as code using Terraform, CloudFormation, CDK or similar
• Linux administration and Docker
• dbt in production, including incremental models and testing
• Workflow orchestration with Airflow, Dagster, Prefect or similar
• Git, CI/CD and code review practice
• Strongly preferred: Apache Iceberg or equivalent, ClickHouse or comparable warehouse, Terraform, Apache Airflow, AWS Glue and Lake Formation, and analytics data modelling
• Advantageous: Databricks, Kafka/Flink/Kinesis/Spark Structured Streaming, event or IoT data, Airbyte, Apache Superset administration, and API or systems integration development

• Full-time remote position
• Occasional international travel for in-person team meetings, field support or training, as needed
• Equal opportunity employment