Imagine this. You need an image of a balloon for a work presentation and turn to an AI text- to- image generator, like Midjourney or DALL-E, to create a suitable image. You enter the prompt(提示词)" red balloon against a blue sky" but the generator returns an image of an egg instead.
What's going on? The generator you' re using may have been" poisoned". What does this mean?
Text- to- image generators work by being trained on large databasets that include millions or billions of images. Some of the generators have been trained by indiscriminately (任意地) scraping online images, many of which may be under copyright. This has led to many copyright infringement(侵害) cases where artists have accused big tech companies of stealing and profiting from their work.
This is also where the idea of" poison" comes in. Researchers who want to empower individual artists have recently created a tool named" Nightshade" to fight back against unauthorised image scraping. The tool works by slightly changing an image's pixels(像素) in a way that confuses the computer vision system but leaves the image unchanged to a human's eyes. If an organization then scrapes one of these images to train a future AI model, its data pool becomes" poisoned". This can result in mistaken learning, which makes the generator return unintended results. As in our earlier example, a balloon might become an egg.
The higher the number of" poisoned" images in the training data, the greater the impact. Because of how generative AI works, the damage from" poisoned" images also affects related prompt keywords. For example, if a" poisoned" image of a Picasso work is used in training data, prompt results for masterpieces from other artists can also be affected.
Possibly, tools like Nightshade can be abused by some users to intentionally upload" poisoned" images in order to confuse AI generators. But the Nightshade's developer hopes the tool will make big tech companies more respectful of copyright. It does challenge a common belief among computer scientists that data found online can be used for any purpose they see fit.
Human rights activists, for example, have been concerned for some time about the indiscriminate use of machine vision in wider society. This concern is particularly serious concerning facial recognition. There is a clear connect ion between facial recognition cases and data poisoning, as both relate to larger questions around technological governance. It may be better to see data poisoning as an innovative(创新的) solution to the denial of some fundamental human rights.