Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)

Turing
Turing
Verified company
RemotePosted Oct 11, 2026

Country eligibility may be restricted

This opportunity may not be available in Unknown.

🇺🇸 US

Competitive /hr

Compensation

Contract

Employment

Senior Level

Experience

Oct 11, 2026

Posted

About this opportunity

About Turing: Turing is one of the world's fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Turing helps leading AI labs improve the reasoning, problem-solving, and decision-making capabilities of large language models (LLMs) through high-quality human feedback, evaluation, and training data. Role Overview: We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models. In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks. This position is ideal for engineers who enjoy code review, debugging, problem-solving, and applying strong software engineering judgment to complex technical scenarios. Your work will directly contribute to measuring and improving the performance of advanced AI coding systems.

Key responsibilities

  • What Does Day-to-Day Look Like?
  • Review and evaluate AI-generated code for correctness, efficiency, maintainability, and adherence to requirements.
  • Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
  • Debug code, reproduce issues, and verify fixes across different programming environments.
  • Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
  • Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks.
  • Identify edge cases, failure modes, and areas where AI systems struggle with software engineering problems.
  • Document findings clearly and provide structured feedback to improve evaluation quality and consistency.
  • Collaborate with project teams to establish quality standards and evaluation methodologies.

Benefits & perks

  • Perks of Freelancing With Turing:
  • Work in a fully remote environment.
  • Opportunity to work on cutting-edge AI projects with leading LLM companies.
  • Offer Details:
  • Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
  • Engagement type: Contractor assignment (no medical/paid leave).
  • Duration of contract: 1 month (expected start date is next week).
  • Location: US only.
  • Referral Bonus: $250.
  • Evaluation Process:
  • Online automated coding challenge for Python and Docker test (RLHF).

Keep exploring

Similar AI roles

View all