Weekend Reading & Selected Links
Happy weekend! Here are some links to things I've been reading or watching that you might also enjoy:
- Episode 1 of my Compute in Australia Series, with Andrew Charlton. At the bottom of this email, I've included three excerpts.
- Episode 2 of my Compute in Australia Series, with Janet Egan. At the bottom of this email, I've included two excerpts.
- 'The Hugging Face attack surprised me', by Ajeya Cotra (one of the independent investigators).
- 'AI Can Help Plan a Bioweapon. Building One is Still Hard.', by Abi Olvera.
- Saloni's new TED talk.
- 'Terra Incognita: The Economics of a Shrinking World', Jesus Fernandez-Villaverde & Patrick Norrick.
- 'The Douthat Decade', Dan Hitchens in First Things.
- Epoch AI's data centers directory.
- 'The Nvidia-sized hole in US GDP statistics'.
- This is what a Jurassic forest may have sounded like.
Thanks, and have a great week ahead,
Joe
Three excerpts from my podcast with Andrew Charlton
1. Announced on the pod: the government's strategy to turn compute into sovereign AI
JOSEPH WALKER: So let's talk about sovereign AI now. I'm going to go out on a limb – tell me if I'm wrong – but I feel like one model you might have is ascending the stack: starting with a domestic compute industry and then parlaying that into sovereign AI. Am I on the right track there?
ANDREW CHARLTON: Yeah. The oldest industrial policy trick in the book is you find what you're good at, and then you find where in the value chain the rents are. And you use the leverage of what you're good at in order to climb up to the part of the stack where the rents are. I think that is industrial policy 101.
2. Do we have a concrete plan for maintaining frontier access?
WALKER: I think my point about there are adversarial domains where having a frontier model really matters still stands. I still don't feel satisfied that we've got a plan to ensure frontier access parity for those kind of things.
CHARLTON: Yeah, well, there are different strategies to get that capability, and you can see different countries around the world pursuing those strategies. One strategy is alignment, where you align with a frontier country, and through that alignment and that closeness, you gain access to their technology. Another one is leverage, where you have some critical input that is required, and you are trading that critical input for access.
WALKER: Like compute for access.
CHARLTON: Like compute for access. Or Taiwan has the chips; Australia might have the energy – some critical input that gives you leverage and reciprocity. Third, you could try and build it yourself. And there are countries around the world that are trying to build their own models. So there are different strategies on how you make sure that you have that capability. I think Australia is pursuing elements of all of those. We have good alignment, great allies. We have critical inputs in compute that will be valuable in the supply chain. And we also have a lot of sovereign capability that's growing, and which we're using our strength in other parts of the value chain to continue to expand.
3. India and inference compute
CHARLTON: I think the thing that we need to understand when we think about Australia's position in global compute is that every company in the world is now basically trying to do the same thing. That is, they are trying to put down compute to meet the fastest growing demand in the world. And one of the areas of the fastest growing compute demand in the world is India. Now, how do you connect India to the United States? Well, you need to have some very long cables that go across oceans, but those cables can't go through the South China Sea. So Australia becomes a pretty important location connecting Singapore, India, the United States. And that's why we've had big investment in subsea cables into Australia. And it creates an opportunity for Australia to be a hub for some quite large global workloads.
Two excerpts from my podcast with Janet Egan
1. How seriously do Australian policymakers take AI?
JOSEPH WALKER: You’ve been in Australia for the last few weeks, speaking almost exclusively with Australians. When you go back to Washington, tell me how you’ll summarise your trip and what you’ve learned to friends. So, for example, is there anything that surprised you about how Australian policymakers are thinking about compute or AI in general? Any differences that you’ve noticed between how US policymakers and Australian policymakers are thinking about those topics? What’s stood out to you?
JANET EGAN: Yeah, great question. I think the first thing that is so distinct between Washington, DC and Australia is how seriously people take AI. Again and again, I found myself having conversations where I’ve assumed that people think AI is going to be a big deal, and so I’ve started the conversation there, and find myself having to go back and retrace earlier discussions to say, “Oh, okay, here is where the pace of progress is heading. Here is the evidence that we’re continuing to see massive breakthroughs and we haven’t yet hit a wall, and here is the evidence that the recipes we have for continuing to improve capabilities are continuing to bear fruit, and so we’re on a trajectory.”
In Washington, DC, no one is… well, very few people are questioning that. It would be odd if someone said, “Oh, I don’t think AI is a big deal. Oh, I think it’s just hype.” But the amount of times I’ve been asked in the Australian context, “Wait, isn’t it just a bubble? Like, surely AI is just all hype in the market, like the companies are talking themselves up.” That is a really big juxtaposition between the two different policy landscapes.
What surprised me in the positive way, though, has been how rapidly Australia has been moving to understand and grapple with these issues. You had Charlton on this podcast most recently, and it’s been amazing to witness the journey of initial conversations happening around AI in Australia to actually delving into the issue of compute at a ministerial level. We’ve seen the Department of Prime Minister and Cabinet set up a new AI taskforce. We’ve seen a growing number of policymakers express urgency at addressing these issues.
So I think we’re on the right trend, but I do think that there are a lot of very senior people in positions in government who still aren’t grappling with the realities of AI progress.
One theory I have for this is that it’s very easy to appear knowledgeable and be sceptical, but it’s actually very costly or hard to be knowledgeable and engaged in the issues – particularly if you’re a leading economist in your field or you are a thought leader in your industry and someone is asking you for takes on AI. It’s a cheap and easy take to have to say, “Look, I’m not sure this is a really big issue. I’m a skeptic.” It’s a much harder thing to do to sound really knowledgeable and informed and engage in those issues. So I think we’re moving in the right direction, but there’s still a way to go.
2. Australia has "a month or months" to sort out copyright
WALKER: Do you want to just give a 30-second summary of how copyright is a barrier to training compute in Australia at the moment?
EGAN: Yeah, sure. So I often see people in government say: look at all these pros in Australia’s positive column – it’s like renewables, trusted jurisdiction – and then, sure, we’ve got this negative copyright, but we’ve got so many pros. Why does it matter? I think the misinterpretation here is that copyright is actually a red line for AI companies.
And if we put our AI company hat on and think about this: so they currently have fair use doctrine in the US, which, yes, is going through court cases, but is overwhelmingly being upheld to say AI training is fair use. And so the court cases where you’ve seen big payouts – they’re not about the data that was trained on, it’s about how that data was procured, through pirated books.
But US fair use doctrine, there’s a couple of central tenets to it. The first is that the use of the material is truly transformative, and across the board everyone says, look, yep, that seems to be ticking that box. But the second one is that it is not undermining a market for licences for that data.
Now, this second component is what is at risk if an AI company comes to Australia and pays for a licence here. Because if you start paying for copyright licences in another jurisdiction, you’re actually creating the very market that can be used to say, “Hey, fair use doctrine should not apply here.”
WALKER: And to be clear, you’re not only paying for licences for Australian data, but data globally, right?
EGAN: Global data, yeah.
WALKER: Because Australian copyright law protects global data.
EGAN: And so essentially, if they take this approach, they could be opening themselves up to fairly uncapped liability for all the AI training that they have done.
WALKER: Have you seen anyone actually unpack the economics here? Is it actually uncapped liability? So the cynic in me would – and this is like an alternative view – is like: okay, the copyright question may be headed to the US Supreme Court, and maybe Anthropic is not so much interested in a solution in Australia as it is in using Australia as a pawn in any potential case. And, you know, there’s a reason that they’re likely to float with a $2 trillion US valuation. They can afford to pay for licences. We shouldn’t take them at their word that this is a red line. That’s the cynical view that some people might have. I don’t necessarily share that view. I don’t have a strong view. I’d love someone to actually unpack the economics of it for me.
EGAN: I haven’t done that, and I haven’t seen it done. That could be really interesting. I guess, though, I am sympathetic… The principled view on this is: it seems insane that these big companies have, like, taken all human creativity and knowledge and used it to create these AI models, for which the creative industries are seeing no benefit. That seems like… I can understand why people have strong views on this.
But the pragmatic point is: if they don’t come to Australia, they can go elsewhere, including just in the US, and it’s about the level of their willingness to accept that risk or not.
The indications to me, from watching this play out over the last six months, has been that this is going to be a red line.
It also seems that they aren’t particularly price sensitive; they’re licence sensitive. And so companies are probably happy to pay into a fund that then gets redistributed to creatives, or at least have money exchange hands that makes it into the right pockets, as long as it’s not through the copyright framework.
WALKER: And on that, there’s a neat proposal by Good Ancestors for a fund and an authority-to-train-in-Australia licence, but the two are disconnected.
EGAN: A training permit. And I think that, in a way, it’s Australia having the ability to ask for concessions that companies wouldn’t be able, wouldn’t want to, or wouldn’t choose to offer the world, but are willing to offer one jurisdiction for the safe harbour.
But the other piece on copyright that hasn’t yet been talked about as much, but I think is really important as well, is that our current copyright law is not good for inference either.
WALKER: Yeah, so we were chatting on the phone last week and you mentioned this, and that was a great surprise to me. I had no idea that it was potentially a barrier to inference.
EGAN: So it’s not currently… And I want to caveat, I’m not a lawyer, but I’ve talked to lawyers about this.
It seems to be a bit of a ticking time bomb. So there’s a section in the Copyright Act that says you’re allowed to make temporary copies of a copyrighted work if it’s part of a technical process, but just short-lived and technical. So an example of this will be when you’re pressing play on a movie on Netflix: it actually already makes a copy to your computer – it downloads it so that you can watch it.
Now, that’s allowed, because it’s time-limited, in-session, and it’s part of the technical process. But there’s actually specific language in the Act that says that if it’s copying a work that would be a copyright infringement in Australia, even if it was trained overseas legally but hasn’t got coverage in Australia – that’s not allowed.
And the other key issue that comes up here is it doesn’t really allow for context holding between sessions as well. So inference is changing. It’s no longer a call and response. People are developing up their own personalised AIs that integrate different materials and have context and memory of these materials, and that’s part of their value offerings. And as we get AI that kind of looks to update model weights in response to learning potentially, or saves details across sessions and has persistent memories of different works – that seems to actually run afoul of Australian copyright law.
So at the moment it’s unclear how folks are approaching this. It seems like there’s this unsolved issue that is going to rise its head at some point, but it seems like an important thing to get right, to give the certainty to allow build-out to happen.
WALKER: Yeah, that’s so interesting. I had not considered that.
Do you know how much of a window we have in order to fix copyright, for training to happen in Australia, before major labs pass us by as a significant place for their AI training?
EGAN: I think it’s in a month or months, rather than a year. And it’s moving much more rapidly than Australian policy tends to move, which is why I’m really hopeful that we can do this and do it right in the near term. Once other jurisdictions become more favourable as well – like, if Canada decides to move first, if that happens, for example – all Australia’s leverage to ask for what it wants in return dissipates.
I think it’s also important to model how AI companies are thinking about AI progress as well. So we mentioned before about recursive self-improvement, which is the idea of technology improving technology. And this isn’t a new phenomenon. We’ve had technologies improving technologies since the Industrial Revolution. But with AI, it’s AI improving AI at machine speed. And so basically capabilities progressing way faster than our ability to monitor, understand, control them.
Now, there’s some estimates from people who are close to this subject that they’re looking at 2027 for that to happen. And I think [major AI labs are] looking to build out in other jurisdictions where they can have strong governance partners and thought partners in how to manage some of these issues too, and that window I think is closing as well.
WALKER: So we’ve got to sort out copyright in the next month or two?
EGAN: That’d be good. I mean, I don’t think there has to be legislation that’s passed in the next month or two. I think there has to be a clear commitment to make this happen.
WALKER: So that decisions can be made.
EGAN: And so that decisions can be made and build-out starts to occur.