
I didn't know that they had unrestricted internet access This makes me trust PostTrainBench less Because more recent models have access to better teacher models, better datasets and better papers / methods for post-training. This inflates model scores of the most recent models. I don't think rerunning every model when a new model is added is necessary, but I think they should rerun a small set of anchor models like every 1-3 months to quantify that score inflation
