Senior Data Engineer

• Design, build, and optimize scalable real-time and batch data pipelines
• Use Python, SQL, PySpark, Spark Structured Streaming, Delta Live Tables, and Medallion Architecture
• Manage Databricks workflows, clusters, and Delta Lake
• Implement Unity Catalog, fine-grained access control, OAuth-based authentication, and enterprise security compliance
• Lead cloud migrations from Oracle, Teradata, and SQL Server to GCS, AWS S3, Azure Data Lake Storage, Snowflake, or Redshift
• Build CI/CD pipelines using Terraform and GitHub Actions
• Automate data pipeline deployments, infrastructure provisioning, and proactive monitoring
• Develop and deploy custom data-processing scripts and data-driven APIs using Python and Flask
• Use Docker for containerized processing
• Evaluate emerging technologies for data-intensive machine-learning tasks
• Collaborate with business stakeholders and cross-functional engineering teams to translate requirements into technical architectures and data-driven insights
• Guide projects and mentor team members

• 4–7 years of experience
• Notice period of 0–15 days; immediate joiners or candidates currently serving notice period preferred
• Hands-on experience with terabyte-scale datasets
• Experience designing distributed data processing solutions
• Advanced hands-on experience in Python, SQL, and PySpark
• Expert-level SQL query optimization for massive datasets
• Strong proficiency in Databricks Workflows, Unity Catalog, Delta Lake, and Delta Live Tables
• Deep familiarity with GCP, AWS S3/Redshift, or Azure ADLS Gen2
• Solid experience with Terraform, GitHub Actions, Docker, and CI/CD best practices
• Practical experience with Python web frameworks such as Flask
• Strong problem-solving, communication, and ownership skills
• Ability to guide projects and mentor team members
• Willingness to work 6:00 PM–3:00 AM IST during US business hours shift

• Culture built on transparency, diversity, integrity, learning and growth
• Environment encouraging innovation and professional and personal growth