AI QA Trainer – Freelance Project

• Challenge advanced language models through model evaluation and testing
• Converse with models on real-world scenarios and evaluation prompts
• Verify factual accuracy and logical soundness
• Design and run test plans and regression suites
• Build rubrics and pass/fail criteria
• Capture reproducible error traces with root-cause hypotheses
• Suggest improvements to prompt engineering, guardrails, and evaluation metrics
• Partner on adversarial red-teaming and automation using Python/SQL
• Build dashboards to track quality changes over time
• Document failure modes to improve model reasoning and reliability

• Expertise in model evaluation, LLM safety, prompt robustness, data quality assurance, multilingual and domain-specific testing, grounding verification, and compliance/readiness checks
• Experience with hallucination detection, factual consistency, prompt-injection and jailbreak resistance, bias/fairness audits, chain-of-reasoning reliability, tool-use correctness, retrieval-augmentation fidelity, and end-to-end workflow validation
• Bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or a related field is ideal
• Shipped QA for ML/AI systems
• Safety/red-team experience
• Experience with test automation frameworks such as PyTest
• Hands-on work with LLM evaluation tooling such as OpenAI Evals, RAG evaluators, and W&B
• Evaluation rubric design
• Adversarial testing/red-teaming
• Regression testing at scale
• Bias/fairness auditing
• Grounding verification
• Prompt and system-prompt engineering
• Test automation using Python/SQL
• High-signal bug reporting
• Clear, metacognitive communication
• Secure computer and high-speed internet as a contractor

• Secure computer and high-speed internet supplied by contractor
• Company-sponsored health insurance and PTO do not apply