Data Architect, Databricks

• Build, optimize, and maintain scalable data pipelines and ETL/ELT processes
• Implement data architectures including data lakes, warehouses, and integration layers
• Implement data quality checks, validation processes, and monitoring systems
• Develop and maintain integrations between data sources, third-party tools, and analytics platforms
• Collaborate with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions
• Monitor and optimize pipeline performance, query efficiency, and storage costs
• Create and maintain technical documentation for data architectures, pipelines, and processes
• Implement security best practices and ensure compliance with data privacy regulations
• Leverage AI tools and automation to enhance data engineering workflows and productivity

• Hands-on Databricks migration with the databricks/migrate tool, including notebooks, jobs, clusters, secrets, and ACLs
• Unity Catalog to Glue Catalog migration experience
• Notebook and job migration, including notebooks, DAG pipelines, and DLT pipelines
• Storage path refactoring from /mnt/ and DBFS references to S3-backed locations
• Data migration from Delta Lake format to Iceberg format on S3, including Delta tables, raw files, and incremental sync
• Streaming experience with Auto Loader and DLT checkpoint re-bootstrap
• event_id deduplication
• Validation using row counts, checksums, sample queries, and ETL runtime comparison
• Recreating secret scopes in AWS Secrets Manager
• Experience with AWS S3, Glue, Secrets Manager, IAM basics, Athena, Lake Formation, and EMR
• Proficiency in Spark SQL, PySpark, Java, Python, and Scala
• Agile pod delivery, including daily ticket updates, standups, and code review
• Databricks Certified Consultant certification (Associate/Professional) preferred
• DBR to EMR on AWS migration experience preferred

• Health and dental insurance
• Meal and food allowance
• Childcare assistance
• Extended paternity leave
• Partnerships with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass
• Profit Sharing and Results Participation (PLR)
• Life insurance
• Continuous learning platform (CI&T University)
• Discount club
• Free online platform dedicated to physical, mental, and overall well-being
• Pregnancy and responsible parenting course
• Partnerships with online learning platforms
• Language learning platform
• Health and Well-being team, inclusion specialists, and affinity groups
• Support and accommodations for people with disabilities