We are looking for Top GitHub Contributors – Agentic Review Project candidates for a project delivered through the hiring partner.
What you'll do
- Design challenging, real-world problems that serve as the foundation for a model. Problems should be constructed to target specific core capability loss failures identified in a leading model.
- Integrate the problems into an Agentic development environment, preparing all necessary components using Python, which include: Detailed Instructions and an overview of the required task, A Golden solution that follows the instructions, The necessary Environment, including datasets, Python libraries, and metadata, A Test notebook containing unit tests that solutions must pass.
- Evaluate the cross model’s performance on the tasks.
- Identify tasks where target model fails to pass all tests, specifically classifying the failure as a logical reasoning failure.
- Analyze the agent’s steps (Agent Trajectory) to observe and extract core capability loss patterns from the model.
What you need
- Highly-skilled GitHub contributors.
- Applicants must have strong expertise in either data science, ML, coding, or a deep quantitative background in frontier STEM.
- Able to engage reliably for 30+ hrs per week (on weekdays).
- Completion of an AI video interview is required.
Expertise
Who you work with
Project and contracting process: the hiring partner. Applications continue on the provider's website.
Pay and hours
This role does not publish a rate, for 30 h/week. The work is remote.
Who can apply from where
The provider has not confirmed country eligibility. Check the original offer before applying. With a profile, Tier1 checks your country and languages against this role and every other before you apply.