AI models could be attacked, flawed by this Hugging Face security issue — security worries add to AI concerns
A conversion tool could be used in serious supply chain attacks
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
There is a way to abuse the Hugging Face Safetensors conversion tool to hijack AI models and mount supply chain attacks.
This is according to security researchers from HiddenLayer, who discovered the flaw and published their findings last week,The Hacker Newsreports.
For the uninitiated, Hugging Face is a collaboration platform where software developers can host and collaborate on unlimited pre-trained machine learning models, datasets, and applications.
Changing a widely used model
Safetensors is Hugging Face’s format for securely storing tensors which also allows users to convert PyTorch models to Safetensor via a pull request.
And that’s where the trouble lies, as HiddenLayer says the conversion service can be compromised: “It’s possible to send malicious pull requests with attacker-controlled data from the Hugging Face service to any repository on the platform, as well as hijack any models that are submitted through the conversion service.”
So, the hijacked model that’s supposed to be converted allows threat actors to make changes to any Hugging Face repository, claiming to be the conversion bot.
Furthermore, hackers can also exfiltrate SFConversionbot tokens - belonging to the bot that makes the pull requests - and sell malicious pull requests themselves.
Are you a pro? Subscribe to our newsletter
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
Consequently, they could modify the model and set up neural backdoors, which is essentially an advanced supply chain attack.
“An attacker could run any arbitrary code any time someone attempted to convert their model,” the research states. “Without any indication to the user themselves, their models could be hijacked upon conversion.”
Finally, when a user tries to convert a repository, the attack could lead to their Hugging Face token getting stolen, granting the attackers access to restricted internal models and datasets. From there on, they could compromise them in various ways, including dataset poisoning.
In one hypothetical scenario, a user submits a conversion request for a public repository, unknowingly changing a widely used model, resulting in a dangerous supply chain attack.
“Despite the best intentions to secure machine learning models in the Hugging Face ecosystem, the conversion service has proven to be vulnerable and has had the potential to cause a widespread supply chain attack via the Hugging Face official service,” the researchers concluded.
“An attacker could gain a foothold into the container running the service and compromise any model converted by the service.”
More from TechRadar Pro
Sead is a seasoned freelance journalist based in Sarajevo, Bosnia and Herzegovina. He writes about IT (cloud, IoT, 5G, VPN) and cybersecurity (ransomware, data breaches, laws and regulations). In his career, spanning more than a decade, he’s written for numerous media outlets, including Al Jazeera Balkans. He’s also held several modules on content writing for Represent Communications.
This new malware utilizes a rare programming language to evade traditional detection methods
A new form of macOS malware is being used by devious North Korean hackers
Arcane season 2 confirms the hit series isn’t just one of the best Netflix shows ever made – it’s an animated legend that’ll stand the test of time
