Given the increasing reliance/uptake of AI models and this year's bonanza of PyPI supply chain attacks: here's an attack that combines the two: Rank-One Model Editing (ROME) attacks + later some thoughts on detecting this particularly evasive supply chain attack.
Attack methodology originally published as a piece of research Locating and Editing Factual Associations in GPT: Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov.
Rome attacks:
Supply chain attack is likely prerequisite to get malicious python to the model / GPU, though there's many avenues for that.
Once a library is injected into PyTorch, it performs surgical weight rewrites (vs retraining) and changes specific outputs without a process being spawned, no new files and no network calls made. (Which obviously makes this rather difficult to catch with security tools living above the driver)
Detection:
There's detection potential via tracing CUDA kernels + in investigating their caller stack.
There's a deep dive blog from cyber org Stealthium that covers GPU-side detection (via eBPF) and hypothesises on further detection posssibilities.
Welcome the communities thoughts? Reckon any of these model corruption attacks are in the wild and not yet been caught/seen?
(((Also while the blog example is soviet propaganda, anybody aware of other nations that like both model development, influence campaigns and rewriting history? Just sayin...)))
+ shout out to the folks at Mithril Security for their PoisonGPT work a while back that explored this threat vector IRL with an upload to HF.