← Todas as vagas

Senior Data Engineer

Sobre a vaga

• Design and implement data ingestion, transformation, and orchestration pipelines on Google Cloud
• Ensure the quality, reliability, and scalability of batch and streaming data flows
• Optimize storage and processing costs in BigQuery, Dataflow, Dataproc, and Pub/Sub
• Collaborate with data scientists and analysts to deliver consumption-ready data

• Data ingestion with Dataflow (Apache Beam) for scalable ETL/ELT pipelines, including extraction from external sources such as Oracle on OCI and on-premises SQL Server
• Real-time ingestion with Pub/Sub and batch ingestion with Storage Transfer, including sources such as AWS S3 buckets
• Integration with Oracle, SQL Server, and MySQL via JDBC, Dataflow templates, or custom extraction
• Experience with BigQuery, including data modeling, query optimization, partitioning, and clustering
• Experience using Cloud Storage as a layered data lake (raw, bronze, silver, gold)
• Dataform or dbt for SQL orchestration in BigQuery, along with dimensional modeling best practices
• Batch and streaming processing with Beam and streaming SQL
• Orchestration with Cloud Composer (Airflow) or Workflows
• Integration with Cloud Functions and Cloud Run for serverless tasks
• Advanced SQL (BigQuery SQL), Python (Dataflow, automation, ETL), and Git with CI/CD
• Familiarity with Dataproc, Bigtable, and Firestore is desirable
• Familiarity with Dataplex and Data Catalog for data documentation and cataloging is desirable
• Familiarity with healthcare systems, including PEP/EMR/EHR, RIS, LIS, and scheduling and referral management systems, is desirable