Skip to content
aitrainer.work - AI Training Jobs Platform
Software Engineering Turing

Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation)

Turing • Remote, India, United States, Bangladesh, Egypt, Kenya, Mexico, Nigeria, Pakistan, Turkey, Ghana

Education

Not stated

Type

Hourly

Pay

$15-35/hr est.

Turing does not post a rate for this role. This is our estimate for similar roles in India. How we estimate

Last checked

8d ago

Checked

Only apply if this role fits your skills and interests.

Send this job to a friend

New roles in Telegram: @aitrainerwork

What we know about this role

Weekly hours
20–40 hrs/week
Timezone overlap
Partial overlap with US Pacific time required
Contract length
4 weeks

About this role

From the Turing listing

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation

About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. About the Role: We are looking for experienced software engineers (tech lead level) who are familiar with high-quality public GitHub repositories and can contribute to this project. This role involves hands-on software engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality Why Join Us? Turing is one of the world’s fastest-growing AI companies accelerating the advancement and deployment of powerful AI systems. You’ll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research. What does day-to-day look like:

  • Analyze and triage GitHub issues across trending open-source libraries.
  • Set up and configure code repositories, including Dockerization and environment setup.
  • Evaluating unit test coverage and quality.
  • Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
  • Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
  • Opportunities to lead a team of junior engineers to collaborate on projects.

Required Skills

  • Minimum 3+ years of overall experience
  • Strong experience with at least one of the following languages:  Ruby
  • Proficiency with Git, Docker, and basic software pipeline setup.
  • Ability to understand and navigate complex codebases.
  • Comfortable running, modifying, and testing real-world projects locally.
  • Experience contributing to or evaluating open-source projects is a plus.

Nice to Have

  • Previous participation in LLM research or evaluation projects.
  • Experience building or testing developer tools or automation agents.

Perks of Freelancing With Turing

  • Work in a fully remote environment.
  • Opportunity to work on cutting-edge AI projects with leading LLM companies.

Offer Details

  • Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST. (We have 3 options of time commitment: 20 hrs/week, 30 hrs/week or 40 hrs/week)
  • Employment type  : Contractor assignment (no medical/paid leave)
  • Location : India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico

Evaluation Process (approximately 75 mins)

  • Two rounds of interviews (60 min technical + 30 min technical & cultural discussion)

Requirements

  • Critical Thinking
  • Problem Solving
  • High School Maths
  • Mathematics
  • Must be eligible to work in one of: India, United States, and more

How long hiring takes

On the platforms we refer people to, half of hires came within about 25 days of applying. Time from applying to hire:

Under 2 weeks
~30%
2 weeks to 6 weeks
~40%
6 weeks to 3 months
~15%
Over 3 months
~15%

Software Engineer interview prep

Questions Software Engineer interviewers often ask. They aren't taken from this listing.

How do you approach designing a system when the requirements are still likely to change?

I design around the parts of the problem I'm confident are stable, like the core data model, and keep the parts most likely to change, like specific business rules, isolated behind clear interfaces. Over-engineering flexibility everywhere just adds complexity to parts of the system that were never going to change.

What's your process for debugging an issue that only reproduces in production, not locally?

I add targeted logging around the suspected area and compare production data or configuration against my local setup, since an issue that's environment-specific usually traces back to a real difference between the two rather than a fundamentally different bug. I avoid making speculative changes without first confirming where the divergence is.

How do you decide when a piece of code needs a test versus when it's low-risk enough to skip?

I prioritize tests for logic with real business consequences if it breaks, or code that's likely to be touched again later by someone unfamiliar with it. Simple, rarely-changed glue code gets less coverage, since the cost of a thorough test there often exceeds the risk it's protecting against.

How do you evaluate a proposed technical solution's trade-offs, like performance versus maintainability?

I start from the actual constraint that matters for this specific system, whether that's genuinely performance-critical or whether maintainability is the bigger long-term cost, rather than defaulting to whichever trade-off I personally prefer. Optimizing for performance the system doesn't need just adds complexity nobody benefits from.

See all 10 Software Engineer interview questions →

Typical contract length

Across 28 Turing Software Engineering listings we've tracked, contracts typically run about 8.4 weeks. Length varies by project. This listing's 4-week contract is shorter than the typical 8.4-week Turing Software Engineering engagement.

Why this role

This Senior Software Engineer role is for Ruby specialists to build verifiable software engineering tasks from public repository histories. The review work checks whether AI-generated Ruby fixes hold up against real repository behavior rather than just looking plausible.

Skills and categories

More jobs in these fields:

Related jobs

Verified reviews

Loading reviews…

Community reviews

Loading reviews…
💬

Share your experience with Turing

Sign in to leave a review

Common questions

How and when does Turing pay contractors?

Monthly, in USD, via Deel, Payoneer, or direct bank transfer. You're engaged as an independent contractor responsible for your own local taxes. Plan your cash flow around a monthly cycle if you're used to weekly payouts elsewhere.

Do I need to be a software engineer to work for Turing?

No, not anymore. Turing built its name matching senior engineers with Silicon Valley companies, but it has since expanded into AGI infrastructure work and now hires non-engineering domain experts, technical writers, and researchers for post-training data annotation and RLHF. A strong analytical background and excellent English matter more than coding ability.

What does task-based AI training work look like?

Short, hands-on tasks such as recording videos, sorting images, rating AI answers or checking data. Most take 5 to 60 minutes each.

Can I choose my own hours?

Usually, yes. You log in when you want, pick up a task, finish it and submit it, with no meetings. Other people work from the same pool of tasks, so some days there may be no work until more is added.

What does Software Engineering work look like for a Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation)?

You check AI answers to Software Engineering questions, correct their errors and write expert reasoning. As a Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation), you're hired for that judgment, so hands-on Software Engineering experience matters.

What specific skills does this listing call for?

Critical Thinking, Problem Solving, High School Maths, and Mathematics are named directly in the listing. If you don't have hands-on experience with these, expect the screening to test for them.

How many hours per week does this role require?

Based on the listing, this role is scoped at 20–40 hours per week, over roughly a 4-week contract. Plan your week around it.

How much does this specific role pay?

The listing doesn't state a rate. The $15-35/hr shown here is our estimate from the role type and location (see /pay-methodology), so treat it as a rough guide and confirm the actual rate with the platform before committing time.

What happens when I click Apply on this listing?

Apply opens Turing's site, where you create an account and follow its sign-up steps.

Can I apply from outside India, the United States, Bangladesh and 7 other countries?

This role is open only to people based in India, the United States, Bangladesh, Egypt, Kenya, Mexico, Nigeria, Pakistan, Turkey, and Ghana. The limit is usually about where the platform can legally hire, so applying from elsewhere rarely leads to an offer. Check the description for tax or right-to-work rules before you apply, since they can differ by country.