AI Research Scientist – Scientific Evaluation, ML/AI

• Critically evaluate AI-generated ML/AI research papers for scientific contributions, technical correctness, methodology, experimental design, and conclusions
• Assess scientific novelty and significance against existing state-of-the-art research
• Evaluate experimental rigor and validity, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies
• Identify methodological flaws, unsupported claims, inconsistencies, or misleading results
• Assess reproducibility and technical soundness using experimental evidence, code, and implementation details where available
• Review and verify relevant academic literature, prior work, citations, and novelty claims
• Evaluate whether AI-generated research meets the standards of top-tier ML/AI conferences such as NeurIPS, ICML, and ICLR
• Provide structured, evidence-based assessments covering strengths, weaknesses, critical technical concerns, and overall scientific merit
• Apply detailed evaluation rubrics consistently with clear, well-reasoned justifications
• Provide expert human judgment on whether research is scientifically valid, genuinely novel, and comparable to leading ML/AI conference work

• PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical field, or equivalent demonstrated research expertise
• At least 3 peer-reviewed research papers at NeurIPS, ICML, and/or ICLR (hard requirement), ideally including first-author or significant research contributions
• Substantial hands-on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original scientific work
• Strong understanding of scientific methodology and experimental rigor, including appropriate baselines, statistical validity, benchmark design, ablation studies, and reproducibility
• Ability to independently assess the novelty, significance, and technical soundness of advanced ML/AI research
• Experience reviewing research papers for top-tier ML/AI conferences or journals, ideally NeurIPS, ICML, or ICLR — strongly preferred
• Familiarity with state-of-the-art ML/AI research and established benchmarks
• Ability to verify scientific claims, including reviewing technical arguments, experimental evidence, citations, and code implementations where relevant
• Strong analytical and scientific writing skills
• Ability to apply detailed evaluation guidelines and scoring rubrics while exercising independent expert judgment
• Exceptional attention to detail
• CV and brief note describing research background, areas of expertise, and publication record
• Links to at least 3 peer-reviewed publications at NeurIPS, ICML, and/or ICLR
• Relevant academic peer-review experience, if applicable

• Fully remote and flexible — work from anywhere
• Part-time contractor role with flexible hours
• Work on the evaluation of cutting-edge autonomous AI systems capable of conducting end-to-end scientific research
• Apply your research expertise to help assess and advance the scientific capabilities of frontier AI agents