Technical Reviewer - RL Environment Terminal Benchmarking (Agentic AI)

Mercor is hiring a Technical Reviewer on behalf of a leading AI lab to evaluate and refine benchmarking pipelines for reinforcement learning (RL) environments and agentic AI systems. In this role, you’ll be responsible for reviewing environment design, terminal conditions, and evaluation protocols to ensure accuracy, reproducibility, and fairness in benchmarking. You’ll work closely with researchers and engineers to provide technical feedback that strengthens experimental rigor and system reliability.

You’re a great fit if you:

• Have a background in reinforcement learning, computer science, or applied AI research.

• Are experienced with RL environments.

• Understand benchmarking methodologies, terminal conditions, and evaluation metrics for RL tasks.

• Are comfortable reading and reviewing codebases in Python (PyTorch/TensorFlow a plus).

• Have strong critical thinking skills and can provide structured technical feedback.

• Care deeply about experimental reproducibility, fairness, and standardization in agentic AI.

• Are detail-oriented and capable of reviewing both theoretical formulations and implementation details.

Primary Goal of This Role

To review, validate, and improve reinforcement learning environment benchmarking pipelines, ensuring that terminal conditions, evaluation metrics, and system behaviors are robust, reproducible, and aligned with agentic AI research goals.

What You’ll Do

• Review RL environments and evaluate terminal conditions for correctness and consistency.

• Assess benchmarking pipelines for fairness, reproducibility, and alignment with research objectives.

• Provide structured technical feedback on code implementations and documentation.

• Collaborate with researchers to refine evaluation metrics and methodologies.

• Ensure reproducibility by validating results across different runs, seeds, and hardware setups.

• Document findings and recommend improvements for environment design and benchmarking standards.

Why This Role Is Exciting

• You’ll directly influence the reliability of benchmarking in agentic AI research.

• You’ll work on cutting-edge RL environments that test the limits of intelligent agents.

• You’ll help establish standards for evaluation and reproducibility in a fast-moving field.

• You’ll collaborate with researchers shaping the future of agentic AI systems.

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract and Payment Terms

• You will be engaged as an independent contractor.
• This is a fully remote role that can be completed on your own schedule.
• Projects can be extended, shortened, or concluded early depending on needs and performance.
• Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are weekly on Stripe or Wise based on services rendered.
• Please note: We are unable to support H1-B or STEM OPT candidates at this time.

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems. You will be paid competitively, collaborate with leading researchers, and help shape the next generation of AI systems in your area of expertise.