Senior DevOps Engineer – Databricks, CI/CD

• Design, implement, and evolve the Databricks platform’s CI/CD pipeline, ensuring the automated and auditable promotion of artifacts across development, staging, and production environments
• Standardize the packaging and deployment of workloads using Databricks Asset Bundles (DABs), providing reusable and scalable templates for data domains
• Define the version control strategy (Git), branching policies (trunk-based, GitFlow, or equivalent), code review workflow, and platform versioning and release process in the corporate repository (Bitbucket, GitHub, or GitLab)
• Define and implement automated Git flows and quality gates in the pipeline, including unit and integration testing, data contract validation, static code analysis, linting, and minimum code coverage as merge prerequisites
• Ensure project structuring and modularization, package Python libraries by building .wheel files, and publish them to an artifact registry
• Promote software engineering, automation, and governance best practices, serving as a technical reference and supporting data squads with pipeline deployments
• Monitor the health, stability, and observability of CI/CD pipelines and Lakehouse platform artifacts in production

• Solid experience and professional maturity in DevOps or DataOps, with experience designing, implementing, and maintaining end-to-end CI/CD pipelines
• Strong expertise in implementing CI/CD pipelines with Jenkins, including deployment, rollback, and release versioning strategies
• Advanced knowledge of Git version control, branching strategies (trunk-based, GitFlow), pull requests, code reviews, and branch protection
• Advanced software engineering knowledge using Python or Groovy, with hands-on experience structuring and modularizing projects, packaging libraries (.wheel), and publishing them to an artifact repository
• Hands-on experience creating and automating unit and integration tests for data pipelines using Pytest (or an equivalent tool), including mocks, fixtures, and code coverage metrics
• Strong experience with AWS or Azure cloud ecosystems applied to data platform environments, including identity/IAM management, storage, networking, and managed services
• Preferred: Hands-on experience with Databricks Asset Bundles (DABs), Databricks CLI, and Databricks Repos
• Preferred: Experience with legacy Data Warehouse migration projects to a Databricks Lakehouse architecture
• Preferred: Knowledge of pipeline and infrastructure observability and monitoring using tools such as Datadog or similar solutions
• Preferred: Knowledge of Infrastructure as Code (IaC) using Terraform
• Preferred: Experience with Bitbucket Pipelines, GitLab CI, or GitHub Actions

• Position also open to candidates with disabilities
• Remote work