AI MODEL TRAINING
AI trainer for frontier AI labs through the Handshake AI Fellowship — evaluating model outputs, ranking responses, and contributing domain expertise to improve large language model performance across diverse task types.
Frontier
AI Labs
Project
Beet 2.0
RLHF
Training Signal
WHAT IT IS
The Handshake AI Fellowship connects domain experts with frontier AI labs to generate high-quality training data for large language models. Project Beet 2.0 is a specialized engagement in this program — one of the higher-tier projects requiring specific expertise and producing training signals that directly influence model behavior.
The work falls into a category broadly called RLHF — Reinforcement Learning from Human Feedback. When an AI model generates a response, human evaluators assess its quality. Those assessments are used to fine-tune the model, teaching it to produce better outputs over time. The quality of that feedback loop is directly dependent on the quality of the human evaluators.
Working on the training side of AI systems has given me a distinct perspective on how these models are actually built — what they get right, where they fail, and why the prompting and evaluation layer matters as much as the model architecture itself.
WORK INVOLVED
Response Quality Evaluation
Assessing AI-generated responses across multiple quality dimensions: accuracy, completeness, tone, format, and instruction-following. Evaluations require careful reading, domain knowledge to identify factual errors, and judgment about what makes a response genuinely useful versus superficially plausible.
Comparative Ranking
Given two or more model responses to the same prompt, selecting which is better and articulating why. This comparative signal is particularly valuable for training — it doesn't require defining what "perfect" looks like, just which response is more helpful, accurate, or appropriate in context.
Data Annotation & Labeling
Labeling text, structured outputs, and multi-turn conversations with relevant metadata — intent classification, safety flags, hallucination identification, and response category tagging. The labels create structured training signal that the model can learn from at scale.
Prompt & Response Writing
Crafting prompts in specific domains and writing ideal responses that demonstrate the desired model behavior. This work teaches the model both what to produce and how to structure its output — covering edge cases, appropriate caveats, and domain-specific norms.
WHY IT MATTERS FOR MY WORK
Having worked on both sides of AI — deploying Claude in production systems (AI Ticket Triage) and training the models that power those systems — gives me a more complete picture of how to work with AI effectively. Understanding why models behave the way they do informs better prompt engineering, more realistic expectations, and smarter system design.
It also reinforces that the human judgment layer is not going away. The quality of AI outputs in any production system is ultimately bounded by the quality of the evaluation and feedback that shaped the model — and that's a skill set, not just a process.