Data Engineer

• Execute the end-to-end migration of approximately 1,000 datasets to Snowflake
• Upgrade orchestration workflows from Apache Airflow 2 to Airflow 3
• Migrate legacy Hive data structures and datasets to Apache Iceberg using the Snowpark Connect framework/connectors
• Reconfigure, update, and migrate Confluent/Apache Kafka streaming connections across cloud regions
• Adapt existing data pipelines to target Snowflake environments
• Optimize SQL queries using CTEs, window functions, and complex joins
• Manage, test, and debug DAGs and workflow scheduling in Apache Airflow
• Implement testing strategies and conduct hands-on code debugging to ensure data quality and consistency
• Modernize streaming infrastructure across cloud regions and uphold data quality and performance standards
• Collaborate with the Data Platform team on a major data platform modernization initiative

• Minimum 2+ years of hands-on data engineering experience working with large-scale datasets, Apache Airflow, and Apache Spark
• Strong software engineering fundamentals
• Hands-on Python development experience
• Advanced SQL skills, including complex joins, CTEs, window functions, and query tuning
• Proven experience with Snowflake data modeling and schema architecture
• Direct experience building and testing pipelines using Apache Airflow, Apache Kafka, and Apache Spark / PySpark
• Expertise with Apache Iceberg and Apache Parquet data formats
• Experience with AWS S3, including data lakes, bucket partitioning, and lifecycle policies
• Ability to define testing strategies and independently debug complex distributed data pipelines
• Strong communication skills and ability to work independently in a fast-paced environment
• Ability to demonstrate good connectivity to an AWS WorkSpace region by completing the required connectivity test
• Required English for interviews, documentation, and daily communication
• Ability to overlap until at least 12:00 PM Pacific Time

• Fully remote work arrangement
• Full-time, permanent employment
• Client-provisioned AWS WorkSpace