Anthropic, the developers of a large language model (LLM) framework, an AI platform, recently released a studyshowing that it takes as few as 250 malicious documents to “poison” any AI LLM, even the massive ones trained on billions of records.
What does “poisoning” an LLM mean?
In the study’s examples, normal AI prompts that would provide correct information or computer code then produced unreadable gibberish to the same prompts after being exposed to 250 malicious documents. In reality, that gibberish returned by AI could represent far more dangerous manipulative information, such as:
- Providing links to phishing sites instead of legitimate websites
- Giving false information (think politics, stock price manipulation, and actual “fake news”)
- Corrupting code shared with programmers who use AI to check or develop software
That last example in regards to cybersecurity is especially concerning. When developers ask AI for help with coding, they often copy and paste the code examples provided. A “poisoned” AI could slip in malicious code with vulnerabilities, data leaks, or hidden viruses. Those could then get built into other programs used everywhere from casual phone apps to critical flight control systems.
The takeaway:
Whenever you use AI, assume it might have made a mistake and verify the information elsewhere (and not with another AI bot). AI platforms can be powerful tools, like a smart intern who can handle a lot of your businesses workload, but they still need careful supervision to avoid costly mistakes.
____ ____ ____
The Wyoming SBDC Network is hosted by UW with state funds from the Wyoming Business Council and funded, in part, through a cooperative agreement with the U.S. Small Business Administration. Full funding disclosures available at
wyomingsbdc.org/about
All opinions, conclusions, and/or recommendations expressed herein are those of the author(s) and do not necessarily reflect the views of the SBA.
