AI Agent Evaluator

• Evaluate how well AI assistants understand users, use personal context, and deliver helpful conversations
• Write prompts about questions, interests, or needs relevant to you
• Have natural, multi-turn conversations with two AI assistants
• Include questions drawing on information or activity from connected services where relevant
• Compare how effectively each assistant uses available information and connects relevant context
• Assess whether responses feel helpful, natural, trustworthy, and safe
• Evaluate how each assistant responds to feedback and corrections
• Provide clear explanations supporting evaluations
• Complete verification of currently used digital services and recent activity

• Fluency in one of the supported languages/locales: Spanish — Latin America; Arabic; Portuguese — Brazil; Japanese — Japan; Turkish — Turkey; Indonesian — Indonesia; Hindi — India; Korean — Korea; French — France; German — Germany
• Strong reading comprehension and ability to explain feedback clearly
• Attention to detail and sound judgment when comparing conversation experiences
• Willingness to interact with AI assistants using relevant personal context and connected services
• Recent activity across a qualifying combination of digital services
• Completion of project eligibility verification and corresponding expert requirements
• Experience using AI assistants for everyday questions, planning, or research is nice to have
• Previous experience in AI evaluation, user research, or quality assurance is nice to have
• Ability to identify subtle differences in relevance, personalization, and conversation quality is nice to have

• Independent contractor / Part-time arrangement
• Project duration of up to 3 months
• Remote work
• Opportunity to contribute to training and evaluation of frontier AI models
• Potential consideration for other AI evaluation opportunities if not qualified for this project