Software Engineer, LATAM

• Review realistic programming tasks designed for AI evaluation
• Assess the accuracy and structure of various evaluation harnesses
• Walk through thought processes while checking code validity
• Provide actionable feedback on improving task complexity and realism
• Share screen to walk through environments and identify potential flaws or improvements
• Participate in code review, technical discussion, and direct feedback on task structures

• Professional experience as a software engineer or developer
• Located in Brazil or Argentina
• Familiarity with building or verifying coding tasks and test environments
• Comfortable discussing technical architectures and evaluation frameworks
• Direct experience building, reviewing, or testing evaluation harnesses and programming tasks
• Backend development, full-stack engineering, or quality assurance automation experience

• Paid study participation
• $40 per hour compensation