Skip to Content
GenAIAI/ML On-site

AI Engineer - Quality and Post-Training

Job Overview

This role works at the intersection of applied research, quality, and product engineering to make those systems better. It also means bringing emerging research into the product and working closely with product teams and customers, turning learnings from evals and production usage into better outcomes through quality improvements.

Key points

  1. Use eval results and production traces to improve the entire stack, including the agent harness, model routing, prompts, and retrieval…

  2. Work on post-training models with supervised fine-tuning (SFT) and preference- or reward-based optimization (e.g., DPO, PPO, GRPO), including generating the data for training.

  3. Build RL environments for simulating enterprise workflows, including relevant evalsets, harness support, and LLM-based verifiers for hill-climbing and for internal and external benchmarking.

  4. Improve the quality of representative evalsets, LLM judges and agentic verifiers, and calibrate them against human graders so the scores can be trusted in…

  5. Hands-on experience post-training LLMs including adapting open-weight models to specialized tasks.

  6. Experience building evals for AI assistant and agent systems, including generating evalsets, LLM-as-a-judge verifiers…

  7. Strong coding skills in Python and/or Go (TypeScript is a plus), proficiency in your ML framework of choice…

  8. Product-minded, able to turn a vague problem into a concrete experiment, and thrive in a tight-knit, cross-functional team.

Skills

LLMsMachine LearningPythonTypeScriptGitTicketing ToolsInterviewing

Experience: Mid-level · 5+ years

Similar jobs

More open roles like this one