AI Data Analyst

Run ongoing human labeling for priority AI workflows, including response quality, task success, MCP and tool use, end-to-end Cowork-style workflows.

Triage bad-query reports, downvotes, escalations, and routed quality issues; categorize failure modes and feed clean analysis back to partner teams.

Perform qualitative analysis to identify recurring patterns such as hallucinations, retrieval failures, tool-use issues, weak grounding, and poor workflow completion.

Follow and improve labeling guidelines and rubrics so judgments are consistent, realistic, and useful for model and product improvement.

Strong analytical judgment and attention to detail, with the ability to apply nuanced rubrics consistently.

Experience evaluating AI-generated outputs or working with NLP, search, recommendation, or other ML systems.

Familiarity with structured qualitative analysis, basic SQL/data retrieval, and spreadsheet-based workflows.

Clear written and verbal communication skills, with the ability to explain findings to both technical and non-technical audiences.