Humans rank model outputs, a reward model learns those preferences, and the base model is fine-tuned to maximize the reward. RLHF made instruction-following assistants possible.
It also teaches models to sound helpful even when wrong — the reward captures what raters liked, not what was true.