CA could require tech companies to disclose AI training data

Democratic Assemblymember Jacqui Irwin, a former tech insider, is taking on the industry with a far-reaching bill that would require artificial intelligence developers to disclose what data they use to “train” their systems.

“Consumer confidence in AI systems has not grown at the same rapid pace as industry adoption,” Irwin said at a hearing last month. “Many consumers have valid questions about how these AI systems and services are created.”

The concern from Irwin about AI is notable since she may be the Legislature’s top expert on the tech industry and, occasionally, its champion.

As a young engineer at Johns Hopkins University’s applied physics lab, she was assigned to troubleshoot launches of the U.S. Navy’s Trident II nuclear missiles — making her the Legislature’s only actual rocket scientist.

She’s a former engineer for Teledyne Technologies, a global aerospace and tech conglomerate headquartered in Irwin’s hometown of Thousand Oaks. 

She co-chairs the California Legislative Technology and Innovation Caucus and a national legislative task force on AI, cybersecurity and privacy.

She also has been a favorite of Big Tech, once authoring legislation that critics accused of weakening California’s digital privacy protections on behalf of the tech industry to which she had close family ties. At the time, her husband was the chief operating officer of Amazon-owned Ring.

The prominent lobbying group TechNet in 2017 declared her “Legislator of the Year.” 

But now TechNet and nearly every other lobbying group representing major tech companies oppose her latest legislation, Assembly Bill 2013. The influential California Chamber of Commerce is also opposed to the bill, which the state Assembly voted 56-8 to move to the Senate last week.

Last month, the Chamber’s Ronak Daylami told the Assembly Privacy and Consumer Protection Committee, on which Irwin sits, that Irwin’s bill could expose tech firms’ carefully guarded trade secrets.

“While it may not be obvious on its face,” Daylami said, “the expertise and judgment, as well as the actual selection of data and datasets chosen to train a specific AI model, is itself proprietary.”

Would disclosure fend off AI bias?

But Irwin said her bill would give consumers a powerful tool to better understand the emerging technology, which has raised privacy alarms after it was revealed that tech firms used facial recognition, social media posts and copyrighted material such as artwork and news articles to train their artificial intelligence software.

Irwin said the requirement to disclose training data could also help ward against potential biases in the AI software’s decision making. 

She said the issue piqued her interest at a recent meeting of the National Conference of State Legislatures where she heard a doctors’ group discuss using AI in dispensing medication. The problem, she said, was that it wasn’t clear whether such systems had inherent biases since the companies aren’t required to disclose the data they used to train their systems. 

She wondered: What if it was like a clinical drug trial that only tested the medication on white suburban men, instead of a diverse group of patients whose bodies might react differently?

“With these AI medical devices, you really should know what is the group that it was trained on,” she said. 

Putting it more broadly, Hayley Tsukayama, a legislative advocate for the Electronic Frontier Foundation, likened the disclosure requirements to being able to read a list of ingredients that go into a meal.

“The ingredients list is occasionally much easier to parse than trying to taste a dish at the end and trying to figure out what’s in it,” she said.

Irwin owns Amazon and tech stocks 

The AI disclosure bill is hardly Irwin’s first foray into regulating tech since she joined the Assembly in 2014.

Her office provided what it called a “non-exhaustive” list of 13 other tech and cybersecurity bills Irwin has authored, most of which passed. Some of them were also opposed by the tech industry, which has donated at least $288,000 to her campaigns over the years, according to the Digital Democracy database.

Since 2015, Irwin’s votes have aligned with TechNet’s position on bills 28% of the time, according to a Digital Democracy analysis.

Her most controversial tech legislation, though, was a 2019 bill that critics said would have weakened the state’s landmark California Consumer Privacy Act. The law gives Californians legal authority to order tech companies to tell them what personal information they have collected, and customers can tell the companies to delete it and not to sell it. 

At the time, Irwin’s husband, Jon, was the chief operating officer of Amazon-owned Ring, raising the appearance of a conflict of interest given the Privacy Act regulated the company.

Irwin insisted there wasn’t one. She told Politico at the time it was offensive to assume she was working on behalf of her husband’s company, given her professional background and expertise.

Jon Irwin has since left Amazon to become COO of CENTEGIX, a tech company that makes wearable emergency alert devices and security systems for schools and other institutions, according to his LinkedIn page.

The Assemblymember reported to state ethics officials last year the family sold Amazon stock, valued at between $300,000 to $3 million. State ethics officials allow lawmakers to report wide ranges of their stock portfolio value when they file their annual financial disclosure statements. 

Irwin’s disclosure filings show she also acquired at least $60,000 in cryptocurrency, AI and semiconductor investments last year.

In her interview last week with CalMatters, Irwin declined to provide a more precise figure for the Amazon stock sales or address her other recent investments in tech. She said she complied with the state’s ethics disclosure requirements, and that her and her husband’s investments don’t factor into her decision-making process.

“I make every decision based on what’s best for my constituents,” she said. “I don’t need anybody questioning anything that I do, so we are always very careful about every decision.”

Rather, she said she became interested in tech and cybersecurity legislation because her background made her a natural fit for it. And she’s even made it a point to educate her fellow legislators on cybersecurity.

“You can talk to any of my colleagues; most of them, I’ve grabbed their phones and told them, ‘Oh, my God, you know, your phone is tracking you; these apps are tracking you. Let’s turn off location devices and do a two-step authentication,’ ” she said. “In caucus, I get up and tell people how to make their phones more secure.” ’

Tsukayama, the legislative advocate for the Electronic Frontier Foundation, said Irwin definitely knows complicated tech issues as well or better than anyone in the Legislature, even if the digital consumer rights group sometimes opposes her legislation.

“We haven’t always agreed with her,” Tsukayama said, “but it’s rarely, you know, over her misunderstanding how the technology works.”

CalMatters economy reporter Levi Sumagaysay and data reporter Jeremia Kimelman contributed to this story.