Can AI learn what to optimize from examples instead of just copying them? Supervised fine-tuning (SFT) teaches a model by having it imitate expert answers. A new paper introduces PARED, an alignment method that tries to extract a reusable reward from those same demonstrations. PARED compares the expert answers with the model’s own responses across chosen features such as helpfulness, harmlessness, and topic. It uses those differences to learn an explicit reward, which then guides additional reinforcement learning after SFT. Using 4,000 GPT-5.1-generated demonstrations, responses from PARED-trained models were preferred to those from the original SFT checkpoint in 86.2% and 88.4% of non-tied comparisons under two reward-update approaches, as judged by Gemini 2.5 Flash. Demonstrations could end up being useful for more than imitation. They could also help define an objective that the model continues learning from after SFT.