Data Engineer – Mid-Level

• Build batch and streaming ingestion pipelines
• Design and structure data lakes and data warehouses
• Create optimized datasets for ML
• Implement embedding pipelines
• Build vector indexing for RAG
• Ensure data quality, governance, and security
• Optimize storage and processing costs
• Collaborate with AI Engineers to structure feature stores
• Build the infrastructure that enables Machine Learning and Generative AI applications
• Ensure data is organized, reliable, accessible, and scalable for training and inference

• Programming language: Python
• Advanced SQL
• Scala (optional)
• Orchestration and transformation using Apache Airflow, dbt, and Prefect
• Data processing with Spark, Pandas, and PySpark
• Experience with Data Lakes (S3, GCS, Azure Blob)
• Experience with Data Warehouses (BigQuery, Snowflake, Redshift)
• Knowledge of NoSQL databases
• Knowledge of vector databases (Pinecone, Weaviate, FAISS)
• Streaming with Kafka and Pub/Sub
• Infrastructure with Docker and Kubernetes
• Cloud platforms (AWS, GCP, or Azure)
• Data modeling
• ETL / ELT
• Distributed processing
• Scalable data architecture
• Experience with unstructured data (text, logs, PDFs)
• DataOps concepts
• Data versioning and quality

• Meal and/or food allowance of R$950.00 per month on an iFood card
• SulAmérica Medical Plan (100% paid by the company)
• MetLife Dental Plan
• Pet health insurance
• Life insurance
• TotalPass and Wellhub memberships to support physical activities anytime, anywhere
• Birthday Day Off
• Partnership with Sesc
• Partnership with OnHappy (discounts on hotels and flights)
• Partnerships with educational institutions
• Childcare reimbursement
• Work equipment
• Feedback-driven culture and professional development
• Dynamic environment with significant growth opportunities
• Payroll-deducted loan with some of the lowest interest rates
• Variable compensation in recognition of performance, results, and cultural alignment