AI Conversation Evaluation, Research Specialist

• Review AI conversations involving interpersonal relationships, workplace situations, education, family matters, everyday planning, wellbeing, and other real-world decisions
• Evaluate whether the AI gathered sufficient context before providing advice, recommendations, or proposed solutions
• Identify missing information and suggest relevant follow-up questions that could improve the quality of the AI's response
• Assess whether additional context would materially change the recommendation or decision
• Distinguish between load-bearing information that could change the outcome and non-essential or merely descriptive details
• Conduct qualitative reviews using structured evaluation guidelines and document findings consistently
• Identify missed opportunities for clarification, contextual understanding, and information gathering
• Consider relevant personal, psychological, social, financial, practical, and organizational factors when evaluating conversations
• Collaborate with research teams to maintain consistent evaluation standards and high-quality assessments
• Complete assigned tasks accurately within the required project timelines

• At least 3–5 years of direct professional experience in one or more relevant domains
• Licensed Social Work: Master's degree in Social Work (MSW) or an equivalent graduate-level qualification
• Licensed Social Work: LCSW, LICSW, LMSW, or an equivalent applicable social work credential or license
• Experience in client assessment, case review, interviewing, counseling, or related social-work practice
• Counseling, Therapy, or Psychology: Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology
• Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work
• Qualitative Research, HCI, or Communication: MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline
• Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, qualitative coding, or similar research activities
• Ability to analyze conversations systematically and identify missing context or patterns in human communication
• Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields: Relevant professional qualification, credential, or demonstrated subject-matter expertise
• At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway
• Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work
• Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations