Arabic LLM AI Quality Analyst – Remote, $11/hr

About the role

This position is a short‑term contract focused on evaluating and improving the quality of personalized AI responses in Arabic. You will work remotely, collaborating with a team that designs conversational prompts and assesses model outputs for relevance, grounding, and appropriate use of personal data.

Key responsibilities

• Design multi‑turn conversational prompts that incorporate personal context.
• Evaluate AI responses for relevance, grounding, helpfulness, and correct handling of personal data.
• Perform side‑by‑side (SxS) ranking of AI outputs and identify grounding errors, poor inferences, or forced personalization.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data source usage.
• Maintain strict data hygiene by deleting evaluation conversations after review.

Required profile

• Fluent in Arabic (reading and writing).
• Strong experience in data annotation, AI quality evaluation, content moderation, or related fields.
• Analytical mindset with attention to detail and ability to assess nuanced, ambiguous AI responses.
• BS/BA degree or equivalent experience in an analytical discipline.
• Excellent written communication and structured feedback skills.
• Self‑motivated, able to work independently in a remote environment.
• Reliable desktop/laptop with stable internet connection and full‑time availability (8 hours/day) with a 4‑hour overlap with PST.

Required skills

• Arabic language proficiency.
• Experience with data annotation and AI quality assessment.

What we offer

• Short‑term contract (2 months) with immediate start.
• Remote work with flexible schedule.
• Compensation at $11 per hour.