Senior Data Platform Engineer, Microsoft Fabric

• Own the client's medallion data architecture end-to-end on Microsoft Fabric
• Design and build Bronze, canonical Silver, and Gold Delta tables with PAN-safe physical layouts
• Build Fabric Spark feature pipelines for rolling velocity, merchant-level rolling, hierarchical rollup, and cross-merchant features
• Author per-flavor-version Silver adapters, unioned by name and tested against golden fixtures
• Build the idempotent Bronze-to-canonical-Silver MERGE DAG according to Aperia's canonical Fabric internal specification v1.4
• Build Gold materialization pipelines for transaction-risk and merchant-risk grains with score-source stamping and idempotent merge semantics
• Design and implement Direct Lake semantic models for reporting surfaces and performance-test them against representative data volumes
• Configure OneLake namespace layouts, workspace catalogs, and cross-workspace shortcut relationships
• Establish idempotent Delta write patterns using delta-rs from Python containers with segmented parallel writes and a single atomic commit
• Size Fabric capacity, select F-SKUs, perform load testing, and make Reservation versus PAYG decisions
• Partner with the DevOps engineer on Fabric IaC, including Terraform capacity provisioning and Fabric REST workspace bootstrap
• Consult on a ClickHouse contingency if Direct Lake latency is insufficient for client-tier UIs
• Collaborate with a Senior Technical Lead, Senior DevOps engineer, and mid-level developer on platform foundation and risk product delivery
• Work on a platform ingesting transaction and merchant data via SFTP and API, tokenizing sensitive PAN data at the ingestion boundary, and feeding DMN rules and Mastercard Brighterion feeds

• 5+ years data engineering experience
• At least 2 years hands-on Spark development; PySpark preferred
• Deep Delta Lake internals knowledge, including transaction log format, checkpoints, retention settings, VACUUM, OPTIMIZE, deletion vectors, and statistics collection
• Parquet format expertise, including footers, statistics allowlists, row-group sizing, dictionary encoding, and column indexes
• Microsoft Fabric hands-on experience with workspaces, capacities, notebooks, lakehouses, semantic models, Direct Lake, and SQL analytics endpoint
• SQL performance tuning skills for OLAP (Fabric) and OLTP (PostgreSQL) workloads
• Strong Python data engineering skills, including pandas, PyArrow, and delta-rs
• Ability to communicate architecture clearly in writing and work autonomously on well-scoped features
• PCI-scoped data platform experience
• Financial services or payments domain background
• Power BI semantic model authoring using DAX
• Familiarity with published data-standard governance patterns, including logical model vs physical profile, SCD2 dimensions, natural-key idempotency, and extension attribute stores
• OneLake shortcuts and cross-workspace lakehouse patterns
• ClickHouse or similar OLAP engine familiarity
• Terraform for Azure data resources
• Prior experience with fluid or schema-flexible ingestion patterns

• Flexible and remote cooperation options
• International projects with leading global clients
• Travels related to international projects
• Non-corporate atmosphere
• Access to language classes
• Access to knowledge-sharing initiatives
• Access to private healthcare and life insurance
• Access to multisport card
• The availability and terms of individual benefits may vary depending on the chosen form of cooperation