Arabic LLM AI Quality Analyst – Remote (Short‑Term)

About the role

We are looking for a detail‑oriented Arabic LLM AI Quality Analyst to join our remote team on a short‑term contract. You will design conversational prompts, evaluate AI responses, and ensure the model respects personal data usage while delivering helpful, grounded answers.

Key responsibilities

• Design multi‑turn conversational prompts based on personal context.
• Evaluate personalized AI responses for relevance, grounding, and helpfulness.
• Assess correct and incorrect use of personal data in model outputs.
• Perform side‑by‑side evaluation and ranking of AI responses.
• Identify grounding errors, poor inferences, and forced personalization.
• Write clear, structured rationales referencing specific conversation turns.
• Extract and verify model debug information and data source usage.
• Maintain strict data hygiene by deleting evaluation conversations.

Required profile

• Fluent in Arabic (reading and writing).
• Strong experience in data annotation, AI quality evaluation, content moderation, or related roles.
• Analytical thinking with high attention to detail.
• Ability to evaluate nuanced and ambiguous AI responses.
• BS/BA degree or equivalent experience in a relevant analytical field.
• Excellent written communication and structured feedback skills.
• Self‑motivated, able to work independently in a remote setting.
• Reliable desktop/laptop with stable internet connection and full‑time availability (8 hours/day) with required PST overlap.

Required skills

What we offer

• Short‑term contract (2 months) with immediate start.
• Competitive hourly rate of $11 USD.
• Fully remote work with flexible schedule, requiring a 4‑hour overlap with PST.