The National Institute of Standards and Technology (NIST) is relaunch a tool that tests susceptibility Artificial Intelligence (AI) Models are being “poisoned” by malicious data.
The move comes nine months after President Biden’s decision. Executive order on the safe, trustworthy, and secure development of AI, and is a direct response to that order’s requirement that NIST assist with model testing. NIST also Recently released A program that helps Americans use AI without falling into the trap synthetic or AI-generated content and that promotes the development of AI for the benefit of society.
The tool, called DioptraIt was initially launched two years ago and aims to help small and medium-sized businesses and government agencies. With the tool, someone can determine what type of attacks would cause their AI model to perform less effectively and quantify the performance reduction to see the conditions that caused the model to fail.
Also: Beware of AI “model collapse”: How training with synthetic data contaminates the next generation
Why does this matter?
It’s critical that organizations take steps to ensure the security of AI programs. NIST is actively encouraging federal agencies to use AI in a variety of systems. AI models are trained on existing data, and if someone deliberately injects malicious data (for example, data that causes AI to ignore stop signs or speed limits), NIST notes, the results could be disastrous.
For all of AI’s transformative benefits, NIST Director Laurie E. Locascio says the technology carries far greater risks than those associated with other types of software. “These guidance documents and testing platform will inform software developers about these unique risks and help them develop ways to mitigate them while supporting innovation,” she said in the release.
Also: Security guidelines provide the necessary first layer of data protection in the AI gold rush
Dioptra can test multiple combinations of attacks, defenses and model architectures to better understand which attacks may pose the greatest threats, NIST says, and which solutions might be best.
The tool doesn’t promise to eliminate all risks, but it does claim to help mitigate them while supporting innovation. Available for download free.