Legal Rubric & Ground Truth Auditor (Licensed Attorney)
Mercor • Remote
Education
JD
Type
Hourly
Pay
$120-140/hr
Your rate depends on your experience and the task.
Listed
1d ago
Checked
Only apply if this role fits your skills and interests.
About this role
From the Mercor listing
Mercor is seeking experienced licensed attorneys to audit rubrics and ground-truth answers for a legal research dataset used to evaluate advanced AI systems. You will review existing legal research questions, verify the accuracy of model answers, and refine grading rubrics so the dataset reflects current law and sound legal reasoning. This is a short-term, fully remote engagement.
Responsibilities
Audit grading rubrics for accuracy, completeness, consistency, and clarity.
Verify ground-truth answers against controlling statutes, regulations, and case law.
Identify legal errors, outdated authority, missing issues, and ambiguous grading criteria.
Rewrite or correct ground-truth answers and rubric items where needed.
Provide concise, structured written feedback on each item reviewed.
Practice Areas
The dataset spans eight practice areas. Experience in one or more is required, and breadth across several is a plus:
Administrative / Regulatory
Business & Commercial
Civil Litigation
Constitutional / Civil Rights
Criminal
Family
Health
Immigration
Required Qualifications
Juris Doctor (JD) and active U.S. bar admission in good standing.
5+ years of legal practice.
Substantive experience in at least one of the practice areas above.
Strong legal research, analysis, and writing skills.
High attention to detail and the ability to apply consistent judgment.
Preferred Qualifications
Experience across multiple listed practice areas.
Judicial clerkship, appellate, government, or law firm research experience.
Prior experience reviewing or grading legal work product, or with AI / legal-tech evaluation projects.
Why Join?
Apply your legal expertise to shape how AI systems are evaluated for legal research.
Collaborate with a leading AI research organisation.
Requirements
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- JD's degree or equivalent professional experience
- Demonstrated expertise in Legal
How long hiring takes
On the platforms we refer people to, half of hires came within about 25 days of applying. Time from applying to hire:
Talent pool members
When your profile is a strong match, we can put you forward to Mercor. Set up your profile first if you haven't.
Set up your profile →Why this role
At $120-140/hr, this Legal Rubric & Ground Truth Auditor (Licensed Attorney) role asks you to check AI answers in Legal the way a colleague in your field would.
Talent pool
We're light on Legal candidates
We've tracked 279 Legal listings and have few candidates to put forward for them. Set up a profile and we'll consider you for roles like this one.
Set up your profileSkills and categories
More jobs in these fields:
Related jobs
Verified reviews
Community reviews
Leave your review
Common questions
Does it cost money to apply to Mercor?
No, applying to and joining Mercor is free. Mercor charges the client a fee on top of your rate. If anyone asks you to pay to join, treat it as a scam.
Is Mercor for freelancers or full-time contractors?
Mercor places you with one client for a set contract, such as 'Python Tutor for 3 months'. Most roles are steady contract work rather than one-off tasks.
Is AI training work the same as traditional consulting?
It's closer to teaching. You grade the AI's logic, correct its mistakes and write the expert reasoning it lacks, instead of producing work for a client.
Why do expert AI training roles pay well?
The model already knows the basics. It needs help with hard edge cases, the judgment calls a senior professional in the field gets right.
What does Legal work look like for a Legal Rubric & Ground Truth Auditor (Licensed Attorney)?
You check AI answers to Legal questions, correct their errors and write expert reasoning. As a Legal Rubric & Ground Truth Auditor (Licensed Attorney), you're hired for that judgment, so hands-on Legal experience matters.
How much does this specific role pay?
This listing is posted at $120-140/hr, an hourly rate. Where you land in the range depends on your background and the assessment. Pay can change after we check the listing, so confirm the rate on the platform's application page.
What happens when I click Apply on this listing?
Apply opens Mercor's site, where you create an account and follow its sign-up steps.
Do I need a JD to qualify?
This role lists a JD or equivalent. The domain assessment usually matters more, and if you pass it the degree is often secondary. Some platforms check credentials, so list your qualifications accurately.
How soon will I start working after applying to Mercor?
Usually after a few weeks. Applying puts you in Mercor's pool of candidates, and you start once a client, such as an AI lab, picks your profile.