Legal Rubric & Ground Truth Auditor (Licensed Attorney)

Mercor is seeking experienced licensed attorneys to audit rubrics and ground-truth answers for a legal research dataset used to evaluate advanced AI systems. You will review existing legal research questions, verify the accuracy of model answers, and refine grading rubrics so the dataset reflects current law and sound legal reasoning. This is a short-term, fully remote engagement.

Responsibilities

• Audit grading rubrics for accuracy, completeness, consistency, and clarity.

• Verify ground-truth answers against controlling statutes, regulations, and case law.

• Identify legal errors, outdated authority, missing issues, and ambiguous grading criteria.

• Rewrite or correct ground-truth answers and rubric items where needed.

• Provide concise, structured written feedback on each item reviewed.

Practice Areas

The dataset spans eight practice areas. Experience in one or more is required, and breadth across several is a plus:

• Administrative / Regulatory

• Business & Commercial

• Civil Litigation

• Constitutional / Civil Rights

• Criminal

• Family

• Health

• Immigration

Required Qualifications

• Juris Doctor (JD) and active U.S. bar admission in good standing.

• 5+ years of legal practice.

• Substantive experience in at least one of the practice areas above.

• Strong legal research, analysis, and writing skills.

• High attention to detail and the ability to apply consistent judgment.

Preferred Qualifications

• Experience across multiple listed practice areas.

• Judicial clerkship, appellate, government, or law firm research experience.

• Prior experience reviewing or grading legal work product, or with AI / legal-tech evaluation projects.

Why Join?

• Apply your legal expertise to shape how AI systems are evaluated for legal research.

• Collaborate with a leading AI research organisation.

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract and Payment Terms

• You will be engaged as an independent contractor.
• This is a fully remote role that can be completed on your own schedule.
• Projects can be extended, shortened, or concluded early depending on needs and performance.
• Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are weekly on Stripe or Wise based on services rendered.
• Please note: We are unable to support H1-B or STEM OPT candidates at this time.

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems. You will be paid competitively, collaborate with leading researchers, and help shape the next generation of AI systems in your area of expertise.