| Hello, and welcome to this week’s installment of the Future in Five Questions. This week I spoke with Helen Toner, an artificial intelligence researcher and director of strategy at Georgetown’s Center for Security and Emerging Technology. Last summer Toner she was at the center of the imbroglio surrounding Sam Altman’s ouster at OpenAI, as one of the four members of the nonprofit oversight board that voted to dismiss him. Four days later, Altman returned and she was removed from the board of directors. (Two more OpenAI leaders departed this week.) We talked about her proposed approach to “responsibly scaling” AI systems, how evolutionary biology has shaped her view of AI and its capabilities and why government should demand more transparency from AI companies. An edited and condensed version of the conversation follows: What’s one underrated big idea? Responsible scaling policies. It’s this idea proposed by an AI nonprofit called Metr, which can cut through these years-long arguments that you have between people who think AI is going to be really dangerous really soon, and other people who think that's totally crazy. The basic idea behind a responsible scaling policy is that if you're an AI company, you spell out as you’re trying to build smarter and smarter systems which observations about them would make you say, “Oh, hang on, we're actually not ready to build an AI system that can do that.” For example, if you build a system that’s really effective at hacking, or one that was able to deceive its users and navigate out of its testing environment, when would you want to stop and take a beat and revisit? Responsible scaling policy is the structure in which you articulate some of those limits. There’s a tension to navigate with a technology as fast-moving as AI between on one hand, the fact that the place with the best information, the most expertise and the clearest picture of what's happening is inside the companies — and so anyone from the outside trying to guess what’s going to happen next will probably not be able to as well. Voluntary industry commitments are a good first step, because they are making use of that knowledge, but I think they need to be ultimately tied back to some kind of externally verifiable commitment. What’s a technology that you think is overhyped? The branding of generative AI. Lumping all recent AI developments into this category is not that useful, and creates the wrong expectations and the wrong picture. It leads people to two kinds of mistakes. First, it makes it sound like there's more of a break between what came before and what we have now than is true. It’s actually pretty continuous with the systems we've been building since the early 2010s. It also gives people the wrong expectations for what will come next. To me, the biggest example of that is the idea of AI agents, which are systems that will not just chat to you in a window, but you can give them a task, you can give them a goal, and they'll go out and pursue that autonomously and take real actions. All kinds of tech companies, including most of the big AI companies, are trying to build these agents, trying to adjust their large language models so that they can go out and take these actions. From a technical perspective, there's not really a clear boundary between a large language model that's generating text in a chat window versus one that's generating code to interface with a browser to go out and buy something for you, or perform some task. The idea of “AI” being generative AI makes it feel like it would be a big leap to have an AI that is actually doing these things. So I’ve been dismayed at the gusto with which “generative AI” has taken off, with policymakers talking about it constantly, I just think it's the wrong category. Either we should talk about automation or deep learning in general, or we should talk about some particular application. What book most shaped your conception of the future? “The Secret of Our Success” by Joseph Henrich. He's an evolutionary biologist, so it's actually not supposed to be about AI at all. But it's incredibly relevant, and the reason is that lots of people in AI tend to think of intelligence as a property of a single mind, whether that's a biological mind, or a silicon mind or whatever. You can talk about whether a certain mind is more or less intelligent than another. But Henrich’s work is about what he calls cultural evolution, which is about how humanity’s sophistication with science and technology was not about individual inventors who were brilliant and made huge leaps, but instead was about the slow accretion of different small improvements. For me, that dynamic compounding over the millennia up to the point where we have today's level of technology really changes how we should think about AI. Instead of thinking about a single system that is smarter than a single human, you also have to account for all of the build-up of expertise and tacit knowledge and understanding that we have at the civilizational level, and this helps explain how successful large language models have been in contrast to some of the earlier systems, like AlphaGo. The whole deal with large language models is that they are trained on as much of our existing cultural knowledge as we can find to stuff into them in the form of available text datasets, which is very different from an idea that was very popular in AI maybe five years ago, which was whether you could get the right reinforcement learning setup where you have an agent that starts out and it doesn't know anything. It's in this complicated simulated world, and it can kind of bootstrap from nothing and get very, very smart on its own. But that was in isolation from all of the real-world things that we have learned over the centuries. What could government be doing regarding technology that it isn’t? Putting policies in place that will help us manage the uncertainty around AI, rather than just trying to resolve the uncertainty one way or another. The one thing that makes AI so hard to regulate is how little expert consensus there is about where the technology is going, and even about where it is right now. This lack of consensus means that debates about AI policy often go in circles between people who think we need quite draconian regulations quite soon, and other people who think it's all super overblown, and any regulatory actions will be really heavy-handed and squash innovation. Instead I think we need to own up to the fact that we need to be making policy under uncertainty, and looking to policies like requiring more transparency from AI companies. Not saying what they have to do or aren't allowed to do, but just getting more information from them about what they're building, how they're managing risks. What surprised you most this year? I was not expecting Congress to actually manage to pass substantive technology bills. I was surprised both by the TikTok getting all the way through, and I’m also surprised by this new privacy bill from [Democratic Washington Sen. Maria] Cantwell and [Republican Alabama Rep. Mike] Rogers. It’s obviously still TBD whether the privacy bill actually goes anywhere, but the fact that it actually seems to have legs and that privacy experts think it could plausibly get passed is pretty astonishing to me, in a good way. I don't want to get my hopes up that Congress is going to be super functional on tech policy in general, but it’s always nice to have slight positive surprises.
| 
Comments
Post a Comment