AI Scraping Triggers 'Defensive Enclosure' of Global Cultural Heritage
Museums and libraries are restricting public access to collections to combat aggressive commercial AI data harvesting.
The mission of open cultural heritage is facing a critical inflection point as the rise of generative AI drives institutions to lock down their digital archives. This shift toward "defensive enclosure" threatens to undo decades of progress in public access to the world's shared history.
Commercial AI developers have aggressively scraped data from Cultural Heritage Institutions (CHIs), including museums and libraries, to train profit-driven models. This activity has created significant infrastructure challenges, characterized by surges in bot traffic that threaten the economic sustainability of these institutions. In response, several major entities have begun restricting data access to prevent exploitation. Notable examples include the National Library of the Netherlands and Trove, the digital repository of the National Library of Australia, both of which have implemented restrictions to curb AI scraping.
The Erosion of the Cultural Commons
For over twenty years, CHIs have acted as stewards of human history, utilizing open licenses—most notably those from Creative Commons—to ensure that curated datasets remained available to the general public. These institutions operated on a foundation of trust, viewing their collections as a global commons. However, the emergence of generative AI has transformed these public resources into high-value training assets. AI companies often ingest this data without providing transparency, attribution, or any form of value-sharing, effectively privatizing the benefits of publicly funded cultural preservation.
The Paradox of Protection
This tension has created a dangerous paradox for the public interest. While restricting access is a logical defense against corporate exploitation, it simultaneously penalizes the general user. If the world's leading libraries and museums move toward total enclosure to protect their assets from AI bots, the public loses the ability to engage with vital cultural resources. Closing up completely is inefficient at best and harms public interests at worst, risking the collapse of a twenty-year legacy of open heritage.
Seeking a Middle Path
To prevent a total retreat from openness, Creative Commons is developing a new tooling system called "CC Signals." This initiative aims to establish a "middle path" between unrestricted access and total enclosure. By creating standardized, prosocial conditions for AI reuse, CC Signals intends to provide a framework where AI developers can access data under agreed-upon terms that respect the sustainability and mission of the providing institutions.
Industry observers are now watching whether these technical signals can be adopted widely enough to stop the trend of enclosure. The outcome will determine whether the digital age expands the reach of human heritage or walls it off in a battle between public stewards and private AI interests.