Data Architect, Databricks

• Build, optimize, and maintain scalable data pipelines and ETL/ELT processes for analytics and business intelligence
• Implement data architectures, including data lakes, warehouses, and integration layers
• Implement data quality checks, validation processes, and monitoring systems
• Develop and maintain integrations between data sources, third-party tools, and analytics platforms
• Collaborate with Data & Analytics Managers and business stakeholders to understand requirements and deliver technical solutions
• Monitor and optimize data pipeline performance, query efficiency, and storage costs
• Create and maintain technical documentation for data architectures, pipelines, and processes
• Implement security best practices and ensure compliance with data privacy regulations
• Leverage AI tools and automation to enhance data engineering workflows and productivity

• Hands-on Databricks migration with the databricks/migrate tool, including notebooks, jobs, clusters, secrets, and ACLs
• Unity Catalog to Glue Catalog migration experience
• Notebook and job migration, including notebooks, DAG pipelines, and DLT pipelines
• Storage path refactoring from /mnt/ and DBFS references to S3-backed locations
• Data migration execution from Delta Lake format to Iceberg format on S3, including Delta tables, raw files, and incremental sync
• Auto Loader / DLT checkpoint re-bootstrap and event_id deduplication
• Validation using row counts, checksums, sample queries, and ETL runtime comparison
• Recreate secret scopes in AWS Secrets Manager
• Experience with AWS S3, Glue, Secrets Manager, IAM basics, Athena, Lake Formation, and EMR
• Proficiency in Spark SQL, PySpark, Java, Python, and Scala
• Agile pod delivery, including daily ticket updates, standups, and code review
• Databricks Certified Consultant certification and DBR to EMR on AWS migration experience are nice to have

• Premium Healthcare
• Meal voucher
• Maternity and Parental leaves
• Mobile services subsidy
• Sick pay-Life insurance
• CI&T University
• Colombian Holidays
• Paid Vacations
• And many others
• Diverse, inclusive, and safe work environment