Engineering Expert

• Design, develop, and validate complex engineering simulation problems for training and evaluating advanced AI models and coding agents
• Develop realistic, multi-constraint problems across Electrical, Mechanical, and Aerospace Engineering
• Create measurable objectives, validated reference solutions, and objectively verifiable automated grading mechanisms
• Build, configure, execute, and validate simulation-based problem environments using open-source engineering tools
• Develop Python scripts, custom test benches, and utilities to automate simulations and verify results
• Ensure simulation outputs satisfy physical plausibility, dimensional consistency, boundary conditions, convergence requirements, and domain-specific constraints
• Troubleshoot simulation failures, numerical instability, dependency issues, and computational errors
• Evaluate LLMs and coding agents across repeated engineering problem-solving trials
• Analyze execution trajectories, tool interactions, simulator feedback, and intermediate outputs to identify reasoning and tool-use failures
• Calibrate task difficulty and refine benchmarks using empirical model performance and evaluation results
• Collaborate with AI researchers, technical leads, and subject-matter experts to integrate validated benchmarks into AI evaluation workflows

• Master's degree or PhD in Electrical Engineering, Mechanical Engineering, Aerospace Engineering, or a closely related technical discipline
• At least 10 years of practical engineering design experience
• Strong Python programming and scripting skills
• Hands-on proficiency with at least one relevant open-source engineering simulation or design tool, such as ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, OpenModelica, CadQuery, build123d, Cantera, or Gmsh
• Experience developing, executing, debugging, and validating simulation-based engineering workflows
• Familiarity with modern LLMs, AI coding agents, and AI evaluation methodologies
• Ability to analyze model execution logs and identify reasoning, planning, and tool-use failures
• Strong understanding of physical constraints, units, boundary conditions, numerical convergence, and engineering validation
• Excellent analytical, problem-solving, and technical documentation skills
• Ability to work independently and deliver technically accurate results within defined timelines
• Approximately 40 hours per week availability
• At least 4 hours of daily overlap with US Pacific Time
• Weekend on-call availability
• Personal desktop or laptop
• Stable, high-speed internet connection

• Remote work arrangement
• Short-term contractor engagement of up to 24 weeks
• Part-time engagement may be acceptable where agreed
• Weekend on-call availability
• Flexible work location among approved eligible locations
• Opportunity to work on AI model training and evaluation benchmarks