Last night I said I'd release the study on whether cross-model pollination actually helps open autoresearch. Here it is, and my answer is no: the brand of model behind the work you build on does not change whether your submission wins. What matters looks more like the thing @sreeramkannan pointed at in the first place, lots of people coordinating on one public frontier. The question came from @0xkydo 's reply to Sreeram's "anons beat anthropic by 2x" post, that this was "a good case study of how cross modal pollination might produce better results." It is a good question, and the https://t.co/zVaWaF5HEj paper makes a similar observation in prose, that plateaus kept getting broken by people using different models and tools. Nobody had checked it against the data, so I did. The basic setup is: I froze the public Yukon API for 10 leaderboards, including https://t.co/zVaWaF5HEj, which is 30,687 scored submissions. For every submission I recorded the company behind the declared model and worked out which earlier submission it built on, from @yukonresearch 's attribution records, the "Base:" line in notes, and code diffs. Then I asked: when you build on someone else's work, are you more likely to get promoted if their work was made with a different company's model than yours? I wrote the whole analysis down and timestamped it with OpenTimestamps before joining any outcome to the data, so the method could not drift toward an answer. The first thing the data shows is that cross-company building is the normal case, not a special event. Two thirds of the steps where someone builds on another person's work switch vendor (image 2), mostly between Anthropic and OpenAI models in both directions. The raw numbers even lean slightly the other way: 6.1% of submissions that built on another company's model were promoted, against 7.6% that built on the same company's. But raw rates mix up who submits, when, and from what starting point. So the real test compares each participant with themselves, at the same stage of the leaderboard. Done that way and combined across the 10 leaderboards, the odds of promotion are about 2% higher for cross-company building, and the data rule out anything beyond roughly 18% lower or 28% higher (image 1). All nine alternative versions of the analysis I fixed in advance give the same answer. That clears the bar I set for calling the effect negligible on average. One honest caveat: I started with a stricter bar, and a pre-registered power check pushed me to a wider one before I saw any outcomes; under the stricter bar this would be undecided. The part I found most fun is the 843-qubit circuit from @sreeramkannan 's post (image 3). It really is a cross-model step. @jieyilong 's GPT-5 submission at 843 qubits and 11.95M Toffolis starts, by its own note, from gnuchev's Claude Opus 5.5 circuit at 843 qubits and 13.01M, which itself built on two BitWonka circuits from the same day. So pollination happens, and on this one step it produced the number everyone quoted. What the data says is that on average, across thousands of steps, the vendor mix is not what separates the steps that win from the ones that don't. Of course it is possible that this is narrower than it looks. It is an average, and a single leaderboard can still differ. It only sees work that was submitted. It trusts the model each person declared, and some people run several vendors under one label. And it is about models, not people: the https://t.co/zVaWaF5HEj paper's point that different participants break plateaus is a separate claim, and nothing here argues against it. , my own views: - On open leaderboards, which vendor's model made your parent submission does not change your odds of getting promoted, on average. - Cross-vendor building is already the default: two thirds of builds on other people's work switch vendor. - The open swarm's edge more plausibly comes from coordination itself, many independent people restarting from one public best, than from mixing model brands. - To test the causal version, platforms would need to log every attempt and every agent's model, not just one declared line per submission. Paper on Zenodo: https://t.co/h9lRYlJYoB
