Subject-Matter Experts – AI Benchmark Question Writers, Reviewers

• Write original, exam-style questions in the native language and selected domain
• Write short-form questions and long-form connected reasoning tasks graded against a rubric
• Test each question on five AI models: GPT, Gemini, Claude Opus, Kimi and Muse Spark
• Produce questions accepted when at least three of five models answer incorrectly
• Write at least five non-challenge questions per domain that models answer correctly
• Provide the correct answer, reasoning, explanation of difficulty and model errors for every question, translated into English
• Record all work in a structured Google Sheets template
• Review accepted questions against criteria covering form, ambiguity, multi-step reasoning, correctness and reasoning explanation
• Comment on issues and approve or return questions

• Native speaker of Thai, Filipino, Vietnamese, Korean or Russian
• University degree in one of the listed domains (Master's or PhD preferred), and/or professional, teaching or research experience in it
• Strong written English, enough to translate questions, answers and explanations accurately
• Comfortable working with AI chat tools and Google Sheets
• Precise, patient and able to work independently to a deadline
• Exam or test-writing experience is nice to have
• University teaching is nice to have
• Expertise in more than one domain is nice to have
• Prior work in AI training data, evaluation or annotation is nice to have
• Able to complete all work by 2 November 2026
• For each covered domain, deliver 20 accepted questions (10 short-form and 10 long-form), plus at least 5 non-challenge questions

• Question Writers are paid in USD per accepted question, with a higher rate for long-form questions
• Reviewers are paid per reviewed question or hourly
• Flexible hours
• Fully remote work
• Opportunity to work on AI benchmark development and evaluation
• Work may cover one or several domains
• Start as soon as possible after onboarding