We are looking for Senior Software Engineer – LLM Evaluation & Repository Validation candidates for a project delivered through Turing.
What you'll do
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Lead a team of junior engineers to collaborate on projects.
What you need
- Strong experience with at least one of the following languages: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing real-world projects locally.
- Experience with well-maintained, widely-used repos with 500+ stars.
Nice to have
- Experience contributing to or evaluating open-source projects.
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.
Expertise
Who you work with
Project and contracting process: Turing. Applications continue on the provider's website.