Legal Reasoning AI Response Evaluator
Claru • Remote
Education
Not stated
Type
Hourly
Pay
$55-100/hr
Your rate depends on your experience and the task.
Last checked
5d ago
Checked
About this role
From the Claru listing
You'll evaluate AI-generated responses to legal questions — covering case analysis, statutory interpretation, procedural questions, and litigation strategy. Your primary task is assessing whether a model's answer reflects sound legal reasoning, accurate doctrinal knowledge, and appropriate caveats. This isn't about copy-editing; you're judging whether the legal analysis would hold up in front of a practitioner, and flagging anything that could mislead a non-expert relying on the output.
Day-to-day work involves reading a legal prompt, reviewing the model's response, and scoring it across dimensions like analytical rigor, jurisdictional accuracy, and practical usefulness. For litigation-focused tasks, you'll also evaluate whether the model correctly identifies procedural posture, burden of proof, and viable arguments. Feedback is written in structured rubric fields, with an optional free-text section for nuanced critique. Tasks are asynchronous — you work batches on your own schedule through a browser-based annotation platform.
A minimum of 3 years of litigation or substantive legal practice is required. Familiarity with U.S. federal and state court procedure is expected; experience in additional jurisdictions is a bonus. Law professors and clerks are also strong candidates. You do not need to be currently practicing, but you must be able to credibly evaluate whether legal reasoning is sound, not just grammatically correct.
Requirements
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Legal
Why this role
At $55-100/hr, this remote Legal Reasoning AI Response Evaluator role doesn't ask you to move or come into an office.
Talent pool
We're light on Legal candidates
We've tracked 269 Legal listings and have few candidates to put forward for them. Set up a profile and we'll consider you for roles like this one.
Set up your profileSkills and categories
More jobs in these fields:
Related jobs
Verified reviews
Community reviews
Leave your review
Common questions
What makes this one of Claru's higher-paying roles?
Claru bands its desk-based work by how much judgment a task needs. This listing sits in the top band at $55-$100/hr, alongside its RLHF evaluation, medical and 3D vision annotation and red-teaming roles, all of which expect graduate-level or professional depth in the subject and the ability to justify a score in writing. Its entry-tier text labeling sits at $20-$35/hr for comparison.
Is Claru desk work a steady queue or does it arrive in batches?
It comes in batches. You pick up a batch when you want, finish it and submit it, with no meetings. A new batch may not be waiting when you finish one, so plan for it as project income.
Can I work on Claru from outside the United States?
For the desk-based annotation and evaluation roles, yes: they are listed as remote with no country gate. The country restrictions on Claru apply to its video capture listings, which are limited to the specific markets named on each one.
Do I need to install anything to do the work?
No. Claru's annotation and evaluation work runs in browser-based tools, including third-party platforms such as Segments.ai for 3D work and DICOM viewers for medical imaging, so there is no local install to maintain.
Is AI training work the same as traditional consulting?
It's closer to teaching. You grade the AI's logic, correct its mistakes and write the expert reasoning it lacks, instead of producing work for a client.
Why do expert AI training roles pay well?
The model already knows the basics. It needs help with hard edge cases, the judgment calls a senior professional in the field gets right.
What does Legal work look like for a Legal Reasoning AI Response Evaluator?
You check AI answers to Legal questions, correct their errors and write expert reasoning. As a Legal Reasoning AI Response Evaluator, you're hired for that judgment, so hands-on Legal experience matters.
How much does this specific role pay?
This listing is posted at $55-100/hr, an hourly rate. Where you land in the range depends on your background and the assessment. Pay can change after we check the listing, so confirm the rate on the platform's application page.
What happens when I click Apply on this listing?
Apply opens Claru's site, where you create an account and follow its sign-up steps.
Who is behind Claru?
Claru is the contributor-facing brand of Reka AI, which collects human data for frontier AI labs across text, vision, video and robotics. Its board splits into regional video capture work and desk-based annotation and evaluation, and the apply link on this page is an affiliate link that does not change your pay or what you are offered.