← Todas as vagas

Senior Engineer – Data Engineering, Databricks

Sobre a vaga

• Provide technical leadership with minimal oversight to deliver innovative solutions on the Databricks Platform
• Use cloud data lakehouse methodologies, distributed processing engines, event streaming platforms, and modern data technologies
• Participate in the Databricks Center of Excellence
• Build next-generation data platforms for organizations in data and analytics
• Work under a Solution Architect to design and implement components of client data platform solutions
• Participate in design sessions, break down complex development tasks, and complete development items on time
• Contribute to Center of Excellence initiatives by developing Databricks solution accelerators
• Develop and implement data solutions using Databricks
• Collaborate with cross-functional teams to design and optimize batch and streaming data pipelines
• Ensure data quality and efficiency
• Stay current with emerging technologies and Databricks platform features and continuously improve relevant skills

• 5+ years of data engineering experience
• At least 2 years of hands-on data pipeline design and development experience with Databricks
• Experience with Delta Lake, Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative Pipelines), and Unity Catalog
• Proficiency designing and building scalable YAML configuration-driven data pipelines
• Experience with batch, micro-batch, and streaming data ingestion and processing patterns using tools such as Auto Loader
• Experience building complex job/workflow orchestration patterns using Databricks Jobs/Workflows or tools such as Airflow and dbt
• Exposure to data quality, pipeline audit, and observability solutions using Databricks native features or tools/frameworks such as Great Expectations, Collibra, and dbt
• Proficiency with Apache Spark, Presto, and Amazon EMR
• Experience with Amazon Redshift, Snowflake, and Google BigQuery
• Strong programming skills in SQL, stored procedures, and object-oriented programming languages such as Java, Python, and PySpark
• Familiarity with DevOps and CI/CD pipelines using Databricks Asset Bundles, including automated validation and testing
• Exposure to Infrastructure as Code tools such as Terraform is a plus
• Familiarity with NoSQL databases and container management systems
• Exposure to AI/ML tools such as MLflow, prompt engineering, and modern data and AI agentic workflows
• Databricks Data Engineering Associate and/or Professional certification is ideal
• Multiple Databricks project delivery experience
• Curious, self-starting mindset and ability to share insights with teams

• Contract engagement for 6+ months
• Start date in Dec/Jan
• Remote work arrangement