Senior Data Engineer – AI, LLM Solutions

• Design and develop enterprise-scale data solutions using Microsoft Fabric, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Databricks
• Build and optimize scalable ETL/ELT pipelines for structured, semi-structured, and unstructured data
• Develop Lakehouse, Data Warehouse, and Medallion Architecture solutions
• Design and implement batch and real-time data processing frameworks
• Lead data modernization, migration, and integration initiatives from legacy platforms
• Develop data foundations for AI, Machine Learning, and Generative AI solutions
• Prepare, transform, and optimize datasets for AI applications, model training, and fine-tuning
• Design and implement RAG architectures and vector search solutions
• Support Azure OpenAI, Microsoft Copilot, and enterprise LLM implementations
• Collaborate with Data Scientists and AI Engineers to operationalize AI and machine learning solutions
• Build distributed data processing solutions using PySpark and Spark SQL
• Develop automated integrations connecting APIs, SaaS applications, databases, and cloud services
• Design reusable data engineering frameworks, components, and standards
• Optimize workloads for performance, scalability, reliability, and cost efficiency
• Implement data quality, validation, monitoring, and error-handling processes
• Implement data governance and cataloging using Microsoft Purview
• Establish data lineage, classification, cataloging, and access controls
• Ensure alignment with security, privacy, compliance, SOC 2, NIST, and CMMC requirements
• Design and support semantic data models for Power BI and enterprise analytics
• Enable self-service analytics and support KPI frameworks, executive dashboards, reporting, and predictive analytics
• Provide technical guidance and mentorship to junior and mid-level data engineers
• Participate in solution architecture, technical design reviews, implementation planning, and presales activities
• Work with clients and stakeholders to translate business needs into scalable technical solutions
• Develop technical documentation, engineering standards, and reusable implementation patterns
• Troubleshoot data pipeline, platform, and integration issues
• Monitor and optimize data workloads and cloud resources
• Support enhancements, upgrades, and ongoing maintenance of data platforms
• Identify improvements in data quality, automation, performance, and operational efficiency
• Stay current with Microsoft Fabric, Azure, AI, Generative AI, and data engineering technologies

• Bachelor’s degree in Computer Science, Software Engineering, Information Technology, Data Science, or a related discipline
• 8+ years of professional experience in Data Engineering, Data Platforms, or Cloud Data Solutions
• 5+ years of hands-on experience with Microsoft Azure Data Services
• Proven experience delivering enterprise-scale data, analytics, and AI solutions
• Strong experience working with modern cloud data platforms and data architectures
• Experience working within consulting, professional services, or enterprise environments
• Strong communication, stakeholder management, and problem-solving skills
• Proficiency with Microsoft Fabric, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage Gen2, Azure Databricks, and Azure Event Hubs
• Knowledge of Data Warehouse, Lakehouse, and Medallion Architecture
• Proficiency in Python, SQL, PySpark, Spark SQL, Kusto Query Language, REST API integrations, and ETL/ELT development
• Experience with SQL Server, PostgreSQL, MySQL, NoSQL databases, data modeling, and optimization
• Knowledge of Azure OpenAI, Generative AI, Prompt Engineering, Retrieval-Augmented Generation, Vector Databases/Vector Search, Machine Learning Pipelines, NLP fundamentals, LLM integration patterns, and AI-ready data architecture
• Experience with Power BI, Tableau, Semantic Models, Executive Reporting, and KPI Frameworks
• Knowledge of Microsoft Purview, Data Governance, Data Classification, Data Lineage, Data Cataloging, Access Control, Enterprise Security Standards, and SOC 2, NIST, and CMMC
• Experience with Azure DevOps, Git, CI/CD Pipelines, Infrastructure as Code concepts, Containerization concepts, and cloud resource optimization
• Ability to design and implement scalable data solutions and understand enterprise data architecture
• Ability to work independently in a remote and cross-functional environment
• Ability to mentor engineers and provide technical leadership
• Strong focus on security, quality, performance, and reliability
• Ability to translate business requirements into technical solutions
• Preferred certifications include Microsoft Certified: Fabric Analytics Engineer Associate, Microsoft Certified: Azure Data Engineer Associate, Microsoft Certified: Azure Data Scientist Associate, Microsoft Certified: Azure Solutions Architect Expert, Databricks Certified Data Engineer, and relevant AI, Machine Learning, or Generative AI certifications

• Remote-first work environment
• Opportunity to work on enterprise-scale Microsoft Fabric and Azure data platforms
• Exposure to AI, Generative AI, Azure OpenAI, and LLM initiatives
• Opportunity to work with modern cloud and data technologies
• Collaborative, innovation-driven culture
• Exposure to diverse enterprise and consulting projects