Malay Audio Generalist Evaluator Expert

Mercor is seeking a Malay Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research lab. In this role, you will work on transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. This is a short-term, structured engagement ideal for candidates with strong academic or analytical backgrounds who are fluent in Malay and English and enjoy translating complex audio and visual information into precise, well-structured text.

Job Responsibilities

Transcribe and Optimise Audio & Video

• Listen to, analyse, and transcribe audio and video content in Malay, following detailed constraints and instructions.

• Produce high-quality written outputs in Malay, with supporting work in English when required.

• Ensure clarity, accuracy, and strict adherence to formatting and stylistic guidelines.

• Capture nuances such as tone, intent, and regional language variations where relevant.

Define and Document Evaluation Standards

• Establish clear expectations for correct and high-quality responses in general consumer audio contexts.

• Develop detailed evaluation rubrics and grading guidelines in Malay and English.

• Document standards to ensure consistency across reviewers and model evaluations.

• Identify edge cases, ambiguities, and linguistic complexities specific to Malay.

Conduct Model Testing and Grading

• Run prompts through language models and assess generated outputs.

• Evaluate responses against predefined criteria for accuracy, completeness, fluency, and instructional clarity.

• Provide structured feedback to improve model performance in Malay audio tasks.

Support Benchmarking and Quality Assurance

• Participate in QA and review cycles to ensure tasks, rubrics, and outputs meet Mercor’s quality bar.

• Maintain consistency and reliability before datasets are integrated into official benchmarks.

• Collaborate with project leads to resolve ambiguities and improve task design.

Minimum Qualifications

• Strong writing, editing, and critical thinking skills.

• Ability to work independently, manage time effectively, and meet deadlines.

• Native or near-native fluency in Malay (spoken and written) and professional fluency in English.

• Ability to accurately transcribe and analyse Malay audio content across general consumer contexts.

• Available to commit 10–20 hours per week.

Preferred Qualifications

• College students or recent graduates.

• Background in linguistics, humanities, social sciences, journalism, or technical disciplines.

• Prior experience with transcription, annotation, localisation, evaluation, or research workflows in Malay.

• Familiarity with dialectal variations, regional speech patterns, and contemporary usage in Malay.

• Interest in AI, language models, or applied research environments.

Application & Onboarding Process

• Complete a short AI-led interview (approximately 15 minutes).

• If selected, you will be onboarded and invited to begin project work.

Additional Role Details

• Work in a structured, goal-oriented project environment with clear tooling, guidelines, and support.

• Gain hands-on exposure to real-world AI research and evaluation workflows.

• Contribute directly to benchmarking and improving advanced multilingual language models.

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract and Payment Terms

• You will be engaged as an independent contractor.
• This is a fully remote role that can be completed on your own schedule.
• Projects can be extended, shortened, or concluded early depending on needs and performance.
• Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are weekly on Stripe or Wise based on services rendered.
• Please note: We are unable to support H1-B or STEM OPT candidates at this time.

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems. You will be paid competitively, collaborate with leading researchers, and help shape the next generation of AI systems in your area of expertise.