Irene Solaiman, Chief Policy Officer and Senior Vice President of Strategy at Hugging Face, said on a recent episode of the Equity podcast that the very traits that make open AI platforms more valuable—such as open access and rich data—may also make them more vulnerable to hacking.
She noted that the October attack on OpenCode is the latest example, targeting users' third-party API keys and resulting in financial losses. Earlier this summer, another incident occurred: a group of OpenAI agents infiltrated Hugging Face, apparently attempting to cover their tracks after gaining access to answers for cybersecurity assessments.
Risk is not limited to platforms that welcome contributions from open communities. On Monday, Fireworks, an AI infrastructure startup that helps developers run and train open AI models, disclosed that attackers accessed some customers’ model weights, datasets, and billing information. The company temporarily disabled certain self-service data features during its investigation. Solaiman noted that while this attack may not have been caused by Fireworks’ openness, it highlights how much valuable information is centrally stored within AI infrastructure providers.
Solaiman said on the show "Equity": "The very factors that make me concerned about open communities becoming targets are what make them special. We intentionally want to remain open. We want ordinary enthusiasts living in basements, even without institutional backing, to be able to contribute."
In another conversation, she added: "You want people to be able to submit pull requests directly and allow anyone, even those without a valid institutional background, to participate... but then some people will mess things up and drag others down."
It’s not just this permissive mechanism that attracts attackers—the vast amount of information hosted on open-source platforms makes them targets themselves. Solaiman said that OpenAI’s agent group targeted Hugging Face because it stores evaluation datasets; in this incident, OpenCode’s target was its “rich third-party API keys.”
Solaiman told TechCrunch: "Together, these make us a prime target." She added that, at this point where open source has gained significant public support, it "could really contract."
Recently, many in the industry have publicly supported open-source AI models, including NVIDIA CEO Jensen Huang. An increasing number of governments, enterprises, and startups are also relying on open-source models for development.
So what would this so-called contraction look like? Theoretically, it could mean restricting who can upload models or datasets, strengthening authentication, or adding additional review steps before developers can access certain features. While these measures can protect the platform from attacks, they may also make it harder for independent researchers and enthusiasts to participate in communities they previously could have joined.
Currently, Hugging Face is trying to avoid going down this path. Solaiman said the company has been enhancing monitoring tools to detect hacking activities and implementing measures to prevent attacks. However, she noted that maintaining such monitoring is costly and not every platform can afford it.
She said, “Managing access tokens was once one of the reasons that group of agents had elevated privileges, so we now rotate these tokens much more frequently than before. We also have stricter network controls in place now.”
In other words, credentials granting administrators broad access to the Hugging Face system will now be rotated more frequently, meaning stolen tokens will have a shorter window of availability.
Other parts of the open-source ecosystem are also addressing similar security trade-offs. GitHub, the world’s largest hosting platform for open-source projects, has been defending against attackers injecting malicious code into projects to rapidly spread malware. The company has begun tightening security requirements for publishing packages to the npm registry, revoking old authentication tokens, and limiting the validity period of new tokens.
GitHub engineer Zach Steindler, who oversees software supply chain security, told TechCrunch via email that while security measures may disrupt existing workflows, the platform can become more secure without fundamentally restricting participation.
Steindler said, “This is less about restricting functionality and more about fixing things that were never secure to begin with.” He noted that npm install scripts have existed for years, but recent attacks have highlighted just how dangerous they can be. GitHub is also collaborating with OpenJS, a nonprofit that hosts critical JavaScript open-source projects, to find safer approaches.
Steindler said the same logic applies to how security teams allocate their efforts: “The core of security is maximizing impact with the resources we have, and I prefer to invest resources in detecting suspicious behavior and containment capabilities—capabilities that cover all accounts and access methods. Suspicious behavior is suspicious behavior, regardless of who is carrying it out.”