Research Evaluation Specialist (PhD or Masters / Researcher / Professor)

• Create original, high-difficulty question-and-answer pairs at the cutting edge of your discipline to rigorously assess AI model capabilities.
• Source, verify, and thoroughly document answers using primary sources and authoritative references, including clear citations and reasoning.
• Design questions demanding advanced reasoning, methodological nuance, or synthesis across multiple sources, ensuring resistance to oversimplified or shortcut solutions.
• Test questions against AI systems, identify items that are not sufficiently challenging, and iteratively increase complexity while maintaining accuracy.
• Write each question and answer with high precision and clarity, eliminating ambiguity and supporting defensible responses.
• Integrate reviewer feedback and consistently uphold all project guidelines and quality standards throughout your deliverables.

• Prior experience in AI is required.
• Prior experience in AI training, data labeling, or model evaluation is required.

• Masters or PhD completed, active candidacy, or equivalent research experience as a specialist, researcher, assistant professor or professor in your domain.
• Demonstrated track record of scholarly research, in-depth subject matter expertise, and familiarity with primary literature.
• Exceptional analytical thinking, close attention to detail, and strong written communication skills in English.
• Experience sourcing and triangulating information from authoritative references.
• Proven self-direction and reliability in delivering expert-level work independently within remote engagements.
• Skill in composing challenging, original, and methodologically sound questions.

RequiredAlso useful

Also listed: attention to detail, analytical thinking.

Project and contracting process: the hiring partner. Applications continue on the provider's website.