Multilingual Audio Evaluator

• Prepare and confirm required tools and access before each live session
• Conduct live, real-time roleplay scenarios with conversational AI agents
• Capture recordings, session IDs, and tool-call or console logs
• Label conversation transcripts according to project guidelines
• Score interactions against structured evaluation rubrics
• Provide clear written explanations supporting scores
• Read, validate, and troubleshoot JSON, identifying structural and formatting issues
• Help improve the quality and reliability of AI conversations across languages

• C1/C2 spoken fluency in at least one target language: Welsh, Dutch, Greek, Italian, French, German, or Portuguese
• Ability to demonstrate language proficiency through a speaking and listening assessment
• Strong JSON skills, including reading, validation, troubleshooting, and identifying structural or formatting errors
• Comfort using digital tools and collecting recordings and technical logs
• Strong attention to detail and ability to follow evaluation guidelines consistently
• Clear written communication and sound judgment when assessing AI interactions
• Location eligibility subject to project location restrictions
• Experience in AI evaluation, audio annotation, or transcript labeling (nice to have)
• Background in technical support, quality assurance, or software development (nice to have)
• Familiarity with conversational AI, APIs, and structured data (nice to have)

• Independent contractor / Project-based
• Remote work
• Project duration of 2–3 months