Member of Technical Staff, Cortex Lab (applied AI research)

• Design and implement evaluation frameworks, benchmarks, and datasets for LLM-powered agents and complex AI workflows.
• Identify, analyze, and prioritize model, agent, and workflow failures in real-world production environments.
• Run rapid experiments to validate hypotheses, measure performance, and improve agent reliability and effectiveness.
• Partner with enterprise customers and researchers to evaluate, deploy, and optimize AI systems across production workflows.
• Develop high-quality training and evaluation data, task specifications, and feedback mechanisms to improve system performance.
• Build Python-based evaluation pipelines, experimentation tools, and lightweight research infrastructure.
• Collaborate with researchers, domain experts, and customer stakeholders to define evaluation strategies and optimization roadmaps.
• Contribute to technical reports, benchmarks, and research insights that inform future system improvements.

• Strong background in applied AI research, research engineering, machine learning, or LLM-based systems.
• Hands-on experience designing and implementing evaluations for AI agents, including tool use, multi-step workflows, and task completion.
• Strong Python programming skills and experience building evaluation pipelines, experimental tooling, or ML infrastructure.
• Experience developing benchmarks, datasets, evaluation metrics, and experimental methodologies for AI systems.
• Ability to analyze agent failures, identify root causes, and translate findings into measurable performance improvements.
• Understanding of LLM behavior, agentic architectures, and common challenges in reliability, robustness, and generalization.
• Ability to translate ambiguous real-world problems into structured experiments and actionable technical solutions.
• Strong collaboration and communication skills, with the ability to work across research, engineering, and customer teams.

• Experience evaluating or optimizing AI agents in production or enterprise environments.
• Familiarity with reinforcement learning, reward modeling, or post-training techniques.
• Experience building research infrastructure, evaluation tooling, or automated experimentation frameworks.
• Prior experience at a frontier AI lab or in an applied AI research environment.
• Contributions to research publications, open-source benchmarks, or AI evaluation frameworks.
• Experience collaborating directly with enterprise customers or technical stakeholders.

RequiredAlso useful

Project and contracting process: the hiring partner. Applications continue on the provider's website.