Behavioral Health Expert, AI Safety and Model Evaluation

• Evaluate conversations between users and AI models across topics such as relationship and family advice, emotional wellbeing, spiritual and metaphysical questions, and unconventional or unfounded beliefs, and assess whether responses are neutral, appropriate and safe.
• Identify patterns of concern, including excessive agreement, taking sides, reinforcing distorted or unfounded beliefs, moralizing, or overstepping into clinical or directive advice, and document them with clear written rationale.
• Develop rubrics, guidelines and reference responses that define a balanced, supportive and appropriately bounded reply, grounded in established practice from counseling and behavioral health.
• Design test scenarios and conversations that probe how models handle sensitive but non crisis topics.
• Collaborate with the lab's researchers and fellow experts to keep evaluation standards consistent, calibrated and well documented.

• Demonstrated ability to remain neutral and nonjudgmental across diverse perspectives, relationships, belief systems and worldviews, and to explain the reasoning behind a professional judgment.
• Working familiarity with concepts such as sycophancy, cognitive distortions, healthy boundaries and client centered approaches such as motivational interviewing.
• Ability to engage reliably for at least 20 hours/week during weekdays.
• Strong written communication skills and the ability to deliver precise, well structured written feedback.

• A background in AI safety, applied ethics, trust and safety or content policy.
• Experience in couples, family or relationship counseling.
• Familiarity with spiritual care, religious or alternative belief communities, or the psychology of misinformation and conspiracy belief.
• Prior experience evaluating, annotating or red teaming AI systems.

Project and contracting process: the hiring partner. Applications continue on the provider's website.