VOTRE PROCHAIN CHAPITRE
Senior Data Engineer
À propos du poste
• Develop, maintain, and optimize notebooks, pipelines, and data transformation processes using Databricks.
• Contribute to the development and evolution of ETL processes, ensuring integration across different data sources.
• Develop Python scripts for data processing, validation, and handling.
• Create, optimize, and maintain SQL queries and routines related to SQL Server databases.
• Build, monitor, and maintain data pipelines, ensuring process availability and reliability.
• Investigate failures, perform troubleshooting, and implement solutions for processing incidents and issues.
• Contribute to the continuous improvement of data quality, governance, and performance.
• Implement development best practices, version control, and monitoring for data processes.
• Collaborate with cross-functional teams to deliver data-driven solutions.
• Hands-on experience with Databricks in developing, maintaining, and optimizing notebooks and pipelines.
• Strong knowledge of Python for data manipulation, processing, and validation.
• Experience with SQL Server, including SQL queries, joins, stored procedures, and query optimization.
• Experience with ETL (Extract, Transform, Load) processes.
• Experience building, maintaining, and monitoring data pipelines.
• Ability to troubleshoot, investigate failures, and resolve problems.
• Knowledge of Azure Data Factory (ADF) for creating, orchestrating, and monitoring pipelines.
• Familiarity with the Microsoft Azure ecosystem and its data storage and processing services.
• Knowledge of PySpark, Apache Spark, and Delta Lake.
• Knowledge of integrating different data sources and processing data.
• Experience with Git and code version control.
• Knowledge of data engineering development best practices.
• Experience optimizing performance and data quality.
• Flexible benefits card – choose how and where to use it.
• Tuition assistance for undergraduate, graduate, MBA, and language courses.
• Certification incentive programs.
• Flexible working hours.
• Competitive salaries.
• Annual performance review with a structured career development plan.
• International career opportunities.
• Wellhub and TotalPass.
• Private pension plan.
• Childcare assistance.
• Medical insurance.
• Dental insurance.
• Life insurance.