Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Country eligibility may be restricted
This opportunity may not be available in Unknown.
🇺🇸 US
Competitive /hr
Compensation
Contract
Employment
Senior Level
Experience
Oct 11, 2026
Posted
About this opportunity
About Turing:
Turing is one of the world's fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Turing helps leading AI labs improve the reasoning, problem-solving, and decision-making capabilities of large language models (LLMs) through high-quality human feedback, evaluation, and training data.
Role Overview:
We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models. In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks.
This position is ideal for engineers who enjoy code review, debugging, problem-solving, and applying strong software engineering judgment to complex technical scenarios. Your work will directly contribute to measuring and improving the performance of advanced AI coding systems.
Key responsibilities
- What Does Day-to-Day Look Like?
- Review and evaluate AI-generated code for correctness, efficiency, maintainability, and adherence to requirements.
- Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
- Debug code, reproduce issues, and verify fixes across different programming environments.
- Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
- Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks.
- Identify edge cases, failure modes, and areas where AI systems struggle with software engineering problems.
- Document findings clearly and provide structured feedback to improve evaluation quality and consistency.
- Collaborate with project teams to establish quality standards and evaluation methodologies.
Benefits & perks
- Perks of Freelancing With Turing:
- Work in a fully remote environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
- Offer Details:
- Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
- Engagement type: Contractor assignment (no medical/paid leave).
- Duration of contract: 1 month (expected start date is next week).
- Location: US only.
- Referral Bonus: $250.
- Evaluation Process:
- Online automated coding challenge for Python and Docker test (RLHF).
Keep exploring
Similar AI roles
Senior Software Code Reviewer
Vetto AI
$40 /task•Remote•Contract
🌍 Worldwide
Coding, Data and Automations Experts
Vetto AI
$50 /hr•Remote•Contract
🌍 Worldwide
Cybersecurity Expert — Talent Pool
Vetto AI
Competitive•Remote•Contract
🌍 Worldwide
Academic Researchers — Talent Pool
Vetto AI
Competitive•Remote•Contract
🌍 Worldwide
Create job alert
Receive notifications when similar opportunities are posted.
Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Turing
Alert criteria
Title: Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Categories: Coding, AI Training, AI Evaluation, Programming
Countries: US
Report this job
Help AI Seeker keep the job marketplace safe and trustworthy.