SAN FRANCISCO — In a startup hub near Chinatown, a few dozen programmers grabbed light beers and flipped open their laptops for a series of demonstrations on how to get around data scraping blockers. Data scraping is the practice of using bots to extract information from websites — a practice as old as the Web, used by everyone from serious researchers and archivists to content-stealing pirates. As that information has become more valuable over the decades, an arms race has sprung up around data-scraping, with many sites building walls to limit access and others aggressively trolling for ways in. DIGITAL FUTURE DAILY readers: We want to hear from you! What do you like in your afternoon tech newsletter? What do you want more of? Please take a few minutes and fill out this survey. The programmers in the room had their own agenda, trying to find ways to keep public data accessible in the spirit of the original Web. They were also there to figure out a pretty important question: What exactly are the rules around data scraping? It turns out that there aren’t many — in part because government regulators haven’t really focused on this increasingly important issue. Internet data is now becoming a crucial resource in the AI boom, and there are ongoing clashes between major tech platforms that are trying to both hoard and capitalize on that data. Caught in the crossfire are advocates for an open internet, and small-time content creators who want to get compensation for their work. It’s a thorny dilemma: The internet was built on a communal ethos of free-flowing information, but that has also allowed AI companies to hoover up content for free, and make immense profits from it. Legally speaking, they’re largely doing it in a void. Federal law only loosely touches on data scraping, and courts have been slow to develop a body of law around the field. Instead, the industry mostly follows a set of customs and voluntary standards. In the absence of real engagement from lawmakers or other regulators, this meeting in San Francisco is a window into how the 30-year-old self-policing experiment called the World Wide Web has become a fractured, profit-driven domain. Large language models are making this an increasingly high-stakes argument — though they’re just the latest twist in a larger story about how internet data has transformed from the lifeblood of a free-flowing informational ecosystem into a monetization opportunity. The meetup was partly a tech demo, and partly a discussion on how to update those standards and influence the process. “If we’re not setting our own best practices, someone else will be doing it for us,” said Jo Levy, chair of the Alliance for Responsible Data Collection, which advocates for preserving open access to internet data.
| 
Comments
Post a Comment