New York is set to become the first state to impose guardrails on “stealth crawlers,” or unauthorized software that trawls news sources to scrape content.
The prohibition on such software bots is meant to curtail the practice of unnamed bots accessing digital services owned by newspapers and broadcast companies, said David Donovan, president of the New York State Broadcasters Association. If approved by Gov. Kathy Hochul (D), the measure would force the companies behind those bots to disclose when they access a news organization’s site.
“Broadcast news websites and digital platforms are besieged by numerous bots gathering information for a variety of purposes,” Donovan said in a statement. “Some bots announce their presence, while many do not.”
It also forbids a stealth crawler from activity to “damage, impair or burden the operation of a covered news site or otherwise cause a news site economic harm,” according to the measure, which was the last bill the state Assembly passed June 5.
The measure (S9934) adds to a host of other technology restrictions the state legislature imposed last week as lawmakers ended their legislative session. The stealth crawler prohibition, along with another measures such as a data center moratorium, a ban on surveillance pricing, and further restrictions on AI chatbots, must receive Hochul’s approval before they become law.
News organizations nationwide have grappled with the rise of AI in repurposing journalistic content, including The New York Times, which is suing technology giant
The New York measure defines a stealth crawler as any software that retrieves, scrapes or otherwise accesses a website, including AI agents. Under the bill, the attorney general’s office would be able to sue companies that fail to disclose such activity. Violations could net civil penalties of up to $15,000 per day.
Two of the state’s media industry groups, the broadcasters association and the New York News Publishers Association, lobbied in support of the measure. Tech and business groups have yet to disclose whether they support or oppose the measure, according to state filings, but generally have opposed other industry regulation on AI chatbots and algorithmic pricing.
So-called “bad bots” access content on websites and overload news organizations with bot traffic, according to bill sponsors.
Those stealth crawlers “have enabled tech companies to free ride off of the hard work of dedicated journalists, all while diverting readers away from the publishers’ own websites,” according to a bill memo from sponsor state Senate Deputy Majority Leader Michael Gianaris (D).