AI Benchmark Engineer, Native Language Specialist – Dutch

• Design, build, and validate Terminal-Bench tasks for multilingual software challenges
• Evaluate coding agents
• Build realistic task environments using Dutch-language datasets and files
• Create prompts and translations in Dutch to identify AI failure points
• Develop robust reference implementations
• Write reliable, deterministic verifier scripts
• Analyze execution logs and calibrate task difficulty from Easy to Very Hard
• Run standard Terminal-Bench configurations against Haiku, Sonnet, and Opus model tiers
• Participate in a four-layer human quality control process: creation, human review, calibration review, and audit
• Work alongside automated LLM-based checks to ensure fairness, grammatical accuracy, and benchmark integrity

• 5+ years of industry experience in software engineering
• Proven track record at leading technology companies and/or graduation from top-tier engineering universities
• Native or near-native fluency in Dutch, with deep understanding of grammar, register, and phrasing rules
• High English proficiency
• Strong proficiency in Python, standard shell scripting, and data processing
• Extensive experience with Terminal/CLI-based development workflows
• Working familiarity with coding agents
• Deep understanding of multilingual text processing pitfalls
• Experience with encoding/decoding robustness and Unicode normalization
• Knowledge of locale-dependent conventions, including collation, casing, and non-Gregorian dates
• Knowledge of text I/O, toolchain interoperability, and safe string operations
• Updated CV in English
• Successful completion of a GenAI assessment
• Reliable availability and commitment; most tasks require a minimum of 2 hours per day or 15–20 hours per week
• Contractors must not be located in regions subject to international embargoes or sanctions
• Contractors are responsible for their own tax obligations

• Flexible schedule with no fixed hours, check-ins, or micromanaging
• Competitive rates
• Prompt payments
• Access to diverse, innovative projects
• Portfolio and skills development across industries and domains
• Global community of linguists, subject matter experts, and language professionals
• No health insurance, paid time off, or retirement contributions are provided
• Work availability fluctuates; hours are not guaranteed