MIT study finds larger AI image models are harder to trace

An MIT study says generative image models trained on large datasets often produce outputs that cannot be traced to any single training image. The researchers call the effect “attribution decay”, and say it grows stronger as the training set gets bigger. MIT’s Computer Science and Artificial Intelligence Laboratory described the finding as a case in which the “connection itself has disappeared”. According to the lab, at sufficiently large scales it can be possible to remove any single image, or even every image by a given artist or person, without changing the generated sample. The paper, published in Nature Communications, defines attributability as the task of locating a part of the training data that can be held responsible for a generated sample. To test that, the authors used ablation, a method that removes exposure to selected training examples without having to retrain a model from scratch each time. Because repeated retraining is costly, the team built a bespoke architecture called a diffusion ensemble. It works like a standard diffusion model while allowing the researchers to switch off the parts that saw a given piece of training data and then compare the resulting output with the original. The study trained 24 diffusion ensembles on subsets ranging from 256 images to 162,770 images, drawn from seven public datasets: MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, CelebA, MetFaces and ArtBench. The researchers measured the difference between factual and counterfactual outputs using both pixel-level geometric distance and semantic distance from OpenCLIP. Across those tests, the relationship was clear: the larger the training set, the smaller the Counterfactual Radius, the study’s measure of how much any one training example could have mattered. The fitted power laws were strong, with R^2 of 0.85 for geometric distance and 0.59 for semantic distance, and the inverse relationship was significant at p < 10^-5 for both. The pattern held when the unit of training data was defined in different ways. For 16 of the 24 ensembles, a unit was a single image. For six, it was all images of one person. For two, it was all images by one artist. In each case, the largest training sets produced significantly smaller Counterfactual Radii than the smallest ones, with p < 10^-4 for single images, p < 10^-53 for single people and p < 10^-22 for single artists. The authors also stress-tested the result by retraining 1,282 diffusion models from scratch at small scale, using MNIST subsets of 256 and 1,024 images. They say attribution decay still appeared there, and also showed up in text-prompt conditioned CelebA models and class-conditioned MNIST models. The finding lands in the middle of wider copyright disputes over AI training data. Semafor noted that companies have trained models on books, articles, photos and other works, prompting lawsuits from Disney, NBCUniversal and DreamWorks against Midjourney, as well as a separate case brought by The New York Times against OpenAI and Microsoft. The study’s authors say the result has implications for fair use, copyrightability and compensation. James Grimmelmann, quoted in the MIT report, said the paper suggests attribution will fail for interesting models and that technologists and courts will need other methods to assess copying.