10的60次方种分子里找药，Isomorphic Labs 要造六个「AlphaFold」

id="podcast-transcript" data-framer-component-type="RichTextContainer" style="transform:none">Max Jaderberg: We have set up the company from day one to really go after this big ambition. This isn’t about developing therapeutics for a particular indication or a particular target. It’s really thinking about how do we create a very general drug design engine with AI, something that we can apply to not just a single target or even a single modality, where we can apply this again and again across any different disease area. And that’s what we’re stepping towards at the moment. Stephanie Zhan: Today we are excited to welcome Max Jaderberg to the show, Chief AI Officer of Isomorphic Labs, which launched out of DeepMind with a goal of revolutionizing drug discovery using AI. Last summer, they released Alphafold 3, a stunning breakthrough that allows us to model not just the structure of proteins but of all molecules, and their interactions with each other. That led to Demis Hassabis winning the Nobel Prize in Chemistry last year.  Max describes their vision for what a holy grail model for drug design and agents for science look like. He draws parallels to his experiences building AlphaStar and Capture The Flag, and the research directions of building agents in games more broadly. Specifically, with 10 to the power of 60 possible drug molecule structures, we need to build both generative models and agents that can learn how to explore and search through the whole potential design space, Max also describes his vision for what a GPT-3 moment for the field might look like – describing it more akin to AlphaGo’s famous Move37, when we start to see things that exhibit superhuman levels of creativity in AI drug design, and that stun even humans ourselves.  This is one of my favorite episodes yet. Enjoy the show! Stephanie Zhan: Max, thank you so much for joining us today here in London. Max Jaderberg: No, it’s a pleasure to be with you here. Yeah, it’s fantastic. Stephanie Zhan: Awesome timing, too, with the launch of AlphaFold 3, and with Demis winning the Nobel Prize in chemistry, which is a true testament to everything that you and your team have done over the last couple years. Max Jaderberg: Yeah, 2024 was definitely a busy year for us. Lots of big breakthroughs. Nobel Prize was just incredible to see. You know, I think amazing recognition for this seminal piece of work. Key questions in RLStephanie Zhan: Yeah. Well, I’d love to start with talking a little bit about your own personal story. You’ve had an incredible career in the world of deep learning from the very start, authoring many seminal papers while at DeepMind, including for Capture the Flag and AlphaStar—breakthroughs in the world of deep learning. Can you walk us through some of the key questions that you had in your field of research around reinforcement learning at the time? Max Jaderberg: Yes. So at DeepMind, I worked on a whole host of stuff, you know, early days of computer vision and deep generative models. But it was really reinforcement learning that ended up hooking me there. DeepMind was the place in the world to be working on reinforcement learning at that time. And really, the question in our minds was: How can we actually get to a point where we could get an AI that could go off and do any task you wanted it to do? And the dominant paradigm at that point in time was supervised learning. Stephanie Zhan: Yeah. Max Jaderberg: And supervised learning is very different from reinforcement learning. They’re both learning techniques, but supervised learning, you need to know what the answer to your question is, and that’s how you train the model. So in supervised learning, you give an example, and then you supply the model with the answer to that question. Now that can be great if you already know everything about the problem that you’re training this AI to do, this neural network to do. Stephanie Zhan: But most times you don’t. Max Jaderberg: Yeah, I mean, there’s just so many problems in the world where we don’t know what the answer is, we don’t know what the solution is. And if you think about—you know, I think about how I want AI to be applied to the world. Yes, it’s going to be great to be able to apply things where we’re already good as humans here. But really, you know, the big frontier is can we start applying AI to places where, you know, humans don’t know how to do this stuff or there’s a limit to human performance there? And, you know, that’s where reinforcement learning is one of the key tools and has real promise here, because in reinforcement learning, you don’t need to know what the answer to the question is. You just need to be able to say whether the answer that the model gave you was good or not good. Stephanie Zhan: Yeah. Max Jaderberg: Maybe even how good or not good. So this opens up a completely new field of problems to train these models against. And so reinforcement learning, and really starting from what was one of the big breakthroughs of DeepMind in the early days was working on games like Atari. Stephanie Zhan: Yeah. Max Jaderberg: The question was: Okay, so how can we scale this up from the world of Pong and Space Invaders to things that really start to look like real problems in the world? And so there was an amazing track of research as we scaled up these methods. Yeah. Stephanie Zhan: Did you know that Sequoia was the first investor in Atari back in the day? Max Jaderberg: Oh, really? I didn’t know that. Stephanie Zhan: [laughs] Max Jaderberg: That’s incredible. Yeah. No, those Atari games were great fun, actually, to sort of go back and play in the context of, “Hey, we’ve got an agent, and I’m just going to have a game of Pong on the side as well.” Stephanie Zhan: There’s a wonderful wall at Sequoia in our office where we have all these names of legendary IPOs and M&As that have happened. And there’s one, I think it’s called The Pizza Company, and I love asking folks if they know what that is. And it’s actually from Chuck E. Cheese’s, which was an original Sequoia investment at the time. Max Jaderberg: Amazing, amazing. Multiplayer gamesStephanie Zhan: So Capture the Flag and AlphaStar were incredible breakthroughs at the time. Can you share a little bit about what exactly those breakthroughs were, and maybe why you chose those specific games? Max Jaderberg: Yeah. So if you think about the history of AI using video games, why do we use video games at all? Video games are these sort of malleable, perfectly encapsulated worlds that as researchers and scientists, we can manipulate them, we can test out different algorithms in them, we can set up different situations, so the perfect test ground for us to develop new algorithms. And then you can imagine as a RL researcher, as someone who’s, like, thinking about how can we get AI to be as general as possible, you’re always thinking, “Okay, we’ve cracked Atari. How do we get a more complex game?” And the thing that I was personally obsessed with is I want these agents to be able to zero shot, be able to do any task. And this is a slightly different paradigm from what people were doing at the time with training on Atari, where normally reinforcement learning, you think about, here’s a game now you get to train on it and get good at it. Stephanie Zhan: Yeah. Max Jaderberg: And then you apply that same algorithm from scratch, training on different games. Stephanie Zhan: Yes. Max Jaderberg: I’d love a different scenario where instead we train an agent, and then we can lift it and put it on any new task. Stephanie Zhan: Yeah. Max Jaderberg: And that agent will be able to perform well in that task without any more training. Stephanie Zhan: Yeah. Max Jaderberg: And so to do that, what you’re really asking for is generalization over task space. Stephanie Zhan: Yes. Max Jaderberg: And that means you need lots and lots of training tasks. So the training data in this RL for agents becomes tasks. Stephanie Zhan: Yeah. Max Jaderberg: Not images, not pieces of text, but tasks. And so you can imagine you could go and sit and, you know, take a whole game studio and try and hand author, you know, hundreds of different tasks, lots of little mini games in these virtual worlds. And we did that. You know, we were doing lots of that. And then you can think, yeah, we can actually go further than hand authoring. We can procedurally generate these tasks and games generating worlds and maps and different objectives. And we did that. But you keep running into this complexity ceiling that there’s only so much complexity that you can handle or you can design humanly. But that’s where multiplayer games come in. Stephanie Zhan: Yeah. Max Jaderberg: Because as soon as you go from single player to multiplayer, it’s not just the agent playing. You’ve got another player in this game, and that other player or other players can take on many different characteristics and many different behaviors. So every different player, every different strategy that you’re up against changes fundamentally the game and what the agent is trying to do. You know, I go back and think why are people still obsessed with playing chess? You know, why does a professional chess player still keep playing chess? It’s the same game, but it’s actually not because you’re playing completely different opponents day after day and new people into the world. So the game is continually changing.  So multiplayer games and multi-agent games really encapsulate that huge diversity of tasks that you might encounter just from other players being there. And so Capture the Flag was actually one of our first forays into how can we use multiplayer games to really stretch what our reinforcement learning algorithms can do to really force us to think strongly about how we can generalize to new tasks, how we deal with these multi-agent dynamics. So Capture the Flag was a fantastic breakthrough. Really showed that we could get to human-level performance for these multiplayer first person games. Stephanie Zhan: Yeah. Max Jaderberg: And then, of course, Starcraft added on a huge amount of complexity and was sort of the next frontier that we had to go after for this. Stephanie Zhan: You were so early in this that so many of these concepts are very, very relevant today in the world of language. How does it feel to see some of this work continue to be played out? Max Jaderberg: Yeah, it’s brilliant. It’s just fantastic actually. You know, there were so many things that we were talking about. Stephanie Zhan: Seven years ago, yeah. Max Jaderberg: Yeah, yeah. You know, 2015, ‘16, ‘17, ‘18. And to see all of these core fundamental concepts be really useful and really applicable today in the world of large language models, you know, and resulting in performance that we could only really dream about at the time, that’s incredibly satisfying. Finding the right recipeStephanie Zhan: Yeah. So then in your own words, you said that you moved from building toys to then finding real applications. When did you know that you found the right recipe? Max Jaderberg: So, you know, I just love deep learning. I’ve been obsessed with deep learning for 10, 15 years now. And the thing that I love about it is that you have these underlying core concepts, these fundamental building blocks that are somehow incredibly transferable between different application spaces. Stephanie Zhan: Yeah. Max Jaderberg: So it’s the same building blocks that we were using in computer vision in 2012 as we were using in early generative models in language, then reinforcement learning, et cetera, et cetera. So what I was seeing just again and again was this ability to take these core concepts, these same core concepts, take incredible people who understand how to—you know, they’re almost like master chefs of putting these concepts together and these different building blocks together, take a team of incredible people, and go after really, really challenging problems. Problems that you go to conferences at the time, and you talk to leading researchers in the field, and they say, “No, no, no. This is 10 years away.” And in the back of your mind, you know, “Actually, we basically cracked it.” Stephanie Zhan: Wow! [laughs] Max Jaderberg: And I saw that happen again and again and again. You take amazing people, amazing algorithms, amazing compute on really challenging problems, and we can find recipes now to crack so many problems. And so it just got to the point where—and I’ve always been quite obsessed with the application of these methods. I want to see this technology have, you know, real transformative, positive impacts in the world. Stephanie Zhan: Yeah. Max Jaderberg: And so, you know, we need to start actually going after that, and you know, the time has been right for I think a few years now. Working with DemisStephanie Zhan: Yeah. Well, so you’ve now had a decade-long relationship working together with one of the greatest scientists, technologists and founders of our lifetime, Demis. He called you while you were still at Oxford, and then your company, Vision Factory and DeepMind were both acquired by Google back in 2014 around the same time. And that’s when the two of you started to work together now for over 10 years. What was it like, or what has it been like to work with Demis? Max Jaderberg: Yeah, I mean Demis is an incredible person, you know, a real character and a real visionary and, you know, also amazingly human and relatable. And I think that that really inspires people. So, you know, it only takes a five-minute conversation for him to sort of really bleed out the depth of ambition that he thinks about, and just the immediacy of the potential to step towards these ambitions. So I think he has this great ability to inject a lot of energy into, you know, a group of very smart people, get people to see beyond what’s right in front of them. You know, I remember moments sitting—well, standing in the lobby of one of the early DeepMind offices. I think this was the—it was a toast, a celebration we were having for the first Nature paper from DeepMind. Stephanie Zhan: Wow! Max Jaderberg: And Demis was saying, you know, this is actually just going to be the first of dozens of Nature papers. Stephanie Zhan: [laughs] Max Jaderberg: And at the time, this was the first—basically the first machine learning paper in Nature. This was the Atari DQN paper. And the prospect of dozens of Nature papers, you know, it seems a bit far fetched. And actually, he went further and said, “And we’re going to be winning Nobel Prizes as a result of this.” Stephanie Zhan: [laughs] Max Jaderberg: And that was 10 years ago. Stephanie Zhan: Yeah. That’s incredible. Max Jaderberg: The forethought that he has. He’s got what I call, like, one of these rollout minds. Maybe it comes from all of his experience playing chess, but he’s always rolling out into the future. Stephanie Zhan: Ah, interesting. Max Jaderberg: What are the steps now that are going to lead to this big ambition? So yeah, it’s been fantastic. I’ve been working with him for about 10 years now, still work really closely together on Isomorphic Labs. And the ambition is as big as ever. If everything goes rightStephanie Zhan: It’s so interesting to hear that you had this ambition, and that he had this ambition from the very start. And it’s incredible that it’s played out that way. Well, I’d love to talk a little bit about Isomorphic. You’re now embarking on one of the most ambitious missions of our generation: To reimagine drug discovery and drug development with AI. If everything goes right and you realize your vision for Isomorphic, what does the world look like? Max Jaderberg: Yeah, we think really big at Isomorphic. We want to be solving all diseases here, and genuinely that scale. And the point is that this technology that we’re building, and AI as a whole field, is going to be completely transformative in how we understand biology, in our ability to manipulate and craft chemistry to modulate that biology. So we really think about a future where we are solving all diseases, where, you know, AI is not just helping us, you know, discover and create and design new therapeutics, but also just understand so much more about our biological world, about how our cells are working, what are the root causes of disease, and therefore opening up new pathways that we can think about modulating. Stephanie Zhan: Yeah. Max Jaderberg: So we have set up the company from day one to really go after this big ambition. This isn’t about developing therapeutics for a particular indication or a particular target, it’s really thinking about how do we create a very general drug design engine with AI, something that we can apply to not just a single target or even a single modality, but we can apply this again and again across any different disease area. And that’s what we’re stepping towards in the moment. Stephanie Zhan: How does setting out with this ambition of being general change how you built in practice from day one? Max Jaderberg: Yeah, it’s a good question. When I think about some of the status quo of AI in drug design, there’s been a lot of use of machine learning models in chemistry and biology, but I would call a lot of the first generation of this sort of application to be more local models. Stephanie Zhan: Yeah. Max Jaderberg: Where you might have some data about a particular target or about how a particular class of molecules is behaving, and you’ll fit a small multi-layer MLP against this data to help you generate some predictions that lead to your next round of design. Stephanie Zhan: Yeah. Max Jaderberg: This is the complete opposite approach of what we were trying to do. So from day one, we were setting out to create models that generalize across chemistry and across target space. And a key example of this is something like AlphaFold and AlphaFold 3, where this is a model that you can apply to a whole different host of targets. You can apply it to any protein in the proteome, in the universe of proteins. You can apply it to any small molecule that you can think of designing without needing to fine tune it, without needing to fit any local data. Stephanie Zhan: Yeah. Max Jaderberg: And so you can imagine that completely changes the way that chemists can use these models if you don’t need to be adapting this model to every single application. Stephanie Zhan: Yeah. Max Jaderberg: So every single one of our internal research projects—and by the way, when I think about, you know, what we’re going to need to get this breakthrough drug design engine that we’ve been building, we need like half a dozen AlphaFolds. Stephanie Zhan: Wow! Max Jaderberg: AlphaFold is just part of the story. Stephanie Zhan: Wow! Max Jaderberg: So from day one, we’ve been setting up these internal research programs, going after these half dozen problems. We’ve had significant breakthroughs, obviously in AlphaFold and structure prediction, but also in other key areas. And in all of these, these models are general. They can be applied to any target. And then what we’re finding, actually, they can be applied to any modality or lots of different modalities, at least. Stephanie Zhan: Yeah. So that’s the first time I’ve heard you say “half a dozen AlphaFolds.” Can you share a little bit more about what that means? Max Jaderberg: Yeah. So AlphaFold was obviously a massive breakthrough in understanding biomolecular structure. So what is the structure of proteins? And now with AlphaFold 3, structure processes with small molecules and things like DNA and RNA. That’s a fundamental step change. It allows us to get experimental-level accuracy of a really core concept of biochemistry that unlocks a whole bunch of thinking and design work for chemists. Stephanie Zhan: Yeah. Max Jaderberg: But, you know, my comment here is actually we’re probably going to need something like half a dozen more of these sort of breakthroughs. They’re sort of getting to experimental-level accuracy of different core concepts of biology and chemistry. Stephanie Zhan: Yeah. Max Jaderberg: To be able to put this together into something that’s really transformative for drug design. Drug design is really, really hard. Stephanie Zhan: Yeah. [laughs] Max Jaderberg: It’s not just a single problem. It’s not just about understanding the structure of a protein. Stephanie Zhan: Yeah. Max Jaderberg: It’s not even just about designing a molecule that will modulate that protein in the way that you want. Stephanie Zhan: Yeah. Max Jaderberg: You want this molecule to be able to, you know, ideally be taken as a pill, and go through the body and be absorbed in the right way and reach the right cell type. And actually go into the cell and not be broken down by the liver in a certain way. Stephanie Zhan: Yeah. Max Jaderberg: So there’s just so much complexity. Stephanie Zhan: Yeah. Max Jaderberg: To hold onto as a drug designer. Stephanie Zhan: Yeah. Max Jaderberg: And each one of those is like an AlphaFold-level style breakthrough that we’ve been creating. “A holy grail model”Stephanie Zhan: So interesting. Well, I’ve also heard you use the words “a holy grail model for drug design and agents for science.” Can you explain a little bit more about what you mean? Max Jaderberg: Yes. So some of these research areas that we’ve been going after, predicting structure and properties of these molecules, and how all of these biomolecules interact and play out over time, these really are sort of holy grail predictive problems for drug design. And we’ve made some incredible breakthroughs there which have really stunned our chemists and step changed how we do drug design internally at Iso. But what’s, I think, a really interesting thing to think about is that you could create the best possible predictive model of the world, like an experimental level, even better than experimental level model, to predict a particular property about a molecule, for example, to be able to predict the outcome of a real experiment. So we could have a whole suite of those, but that still wouldn’t solve drug design. And the way to think about this is there’s this number 1060, which is perhaps all of the possible drug-like molecules that you could—that could exist. Stephanie Zhan: Wow. Max Jaderberg: That’s maybe a bit—takes into account a lot of things. So we could even reduce that by 20 orders of magnitude, get to 1040. Stephanie Zhan: Yeah. Max Jaderberg: That’s still a lot of things. Stephanie Zhan: Yeah. Max Jaderberg: And even if you had the best predictive models in the world, so let’s say you could screen a billion different molecules, you could go and test a billion different molecules, that’s 109. Stephanie Zhan: Yeah. Max Jaderberg: So, you know, now we’re still, like, 1031 molecules left on the table. Stephanie Zhan: Yeah. Max Jaderberg: So even with the best predictive models, you’re still not even scratching the surface … Stephanie Zhan: Yeah. Max Jaderberg: … of molecular space that you should be exploring. And this is why we need to go beyond just predictive models of experiment, but also models like generative models, like agents that can actually navigate that whole 1040, 1060 space. Stephanie Zhan: That’s so interesting. Max Jaderberg: Using our predictive models obviously to understand how to navigate that, but so we don’t have to exhaustively search. Because we can never exhaustively search the whole universe of molecules, if that makes sense. Stephanie Zhan: That’s so interesting. Max Jaderberg: Just in the same way that AlphaGo couldn’t exhaustively search all of the possible Go moves. Stephanie Zhan: Right. Max Jaderberg: Unlike chess, where you could exhaustively search all possible chess moves. Stephanie Zhan: Yeah. Yeah. Max Jaderberg: But yeah, molecule design is much more like Go than it is like chess. So that’s where generative models come into play—agents that utilize generative models, that utilize search techniques as well as these amazing predictive capabilities to really open up the entirety of molecular space. Now to me, it’s actually still amazing that even without AI, we managed to find drugs in this 1060 space. Stephanie Zhan: Yeah. Max Jaderberg: 1040 space. Stephanie Zhan: Yeah. Max Jaderberg: It just says actually there’s probably a lot of redundancy, there’s a lot of potential designs. Stephanie Zhan: Yeah. Max Jaderberg: You know, if you think about a particular disease indication, a particular target, then there should be many designs that exist that would be good for that, and would be the right sort of product profile for this therapeutic. And I think the real potential here is for these generative models, these agents as well, to be able to search through this space and really uncover that whole potential design space. Stephanie Zhan: That’s so interesting. I think in very simplistic layman terms, you’re both learning and modeling the game, and trying to build the best player to solve different types of games. Max Jaderberg: Yeah. So I mean, you know, I’m incredibly biased by games. I’ve been playing video games since I was a kid, grew up in that world. But, you know, that’s exactly how I think about it. We’ve got to be creating our world models, our models of the biochemical world, our biological world. And then we don’t stop there. Stephanie Zhan: Yeah. Max Jaderberg: We actually then need to be creating agents and generative models that can work out how to explore. Stephanie Zhan: Yeah. Max Jaderberg: How to traverse that, and to basically uncover these amazing needles in the haystack in chemical space, which could be life-changing therapeutics for so many millions of people. The breakthroughs of AlphaFoldStephanie Zhan: I love that. That is our punchline today. [laughs] So AlphaFold 3 is truly groundbreaking. You’ve taken us from being able to model just the structure of a protein to now being able to model the structure of all molecules and their interactions with each other. Can you share a little bit about how we should think about that in terms of the impact in accuracy, in speed and efficiency, and also potentially in being able to explore problem spaces that we couldn’t solve before this? Max Jaderberg: Yeah. So, yeah, AlphaFold 2 was the biggest breakthrough, right? Stephanie Zhan: Yeah. Max Jaderberg: To be able to understand the structure of proteins, and then there was something called AlphaFold 2 multimer, which then allows you to understand not just the structure of proteins by themselves, each individual protein, but the structure of proteins as they come together and what we call complexes, so how these proteins fit together. That opens up and helps us answer a lot of questions in biology, but there’s still a big hop to designing therapeutics. And one of the big classes of therapeutics is what’s called ‘small molecules.’ So these are molecules that are not proteins. These would be things like caffeine or paracetamol, things that more often you can take as a pill. And the way that these therapeutics work, these small molecules, is that they go through the body, they go into the cell, and they actually come and attach themselves to these proteins. These proteins, they’re the fundamental building blocks of life. They form these molecular machines by interacting with other proteins. Stephanie Zhan: Yeah. Max Jaderberg: And so you can imagine that if you have another molecule, your drug, that comes in and attaches itself to a protein over here, then it might disrupt the ability for that protein to interact with another protein, part of its normal machine and day-to-day life. And so you’re modulating the function of that protein with this small molecule. Stephanie Zhan: Yeah. Max Jaderberg: And that’s the essence of drug design and how therapeutics work. And so you can imagine as a chemist, your job—or a drug designer, you’re trying to design a small molecule that’s going to fit to this protein over here and disrupt how it normally functions, or in some cases enhance how it normally functions. And so it’d be really helpful to understand how this small molecule interacts with the protein. Stephanie Zhan: Yeah. Max Jaderberg: What’s the structure that it might make, what are the interactions, these literal physical interactions that are being made? Stephanie Zhan: Mm-hmm. Max Jaderberg: And so that really inspired the creation of AlphaFold 3, where now we have a model that not only predicts the structure of proteins, but how these proteins interact with small molecules. Stephanie Zhan: Yeah. Max Jaderberg: Also other fundamental molecular machine-building blocks, things like DNA and RNA. And this basically opens up the ability to structurally understand, which is a core part of drug design, small molecules. It opens up new classes of targets. There are things like transcription factors, which are proteins that sit on DNA and read DNA. And you can imagine now trying to design a small molecule to change or disrupt the function of something like that. And so to do that, you’d really want to be able to see literally in 3D how this all looks. Stephanie Zhan: Yeah. Max Jaderberg: And if I make changes to my little molecule … Stephanie Zhan: Yeah. Max Jaderberg: … how will that change the way it interacts with this protein in this biomolecular system? So AlphaFold 3 is now very, very accurate. It allows us to answer a lot of these questions purely in silico. Stephanie Zhan: Yeah. Max Jaderberg: Or purely on a computer, where before you would have to go to the lab, literally crystallize this stuff. This can take six months, it can take years, sometimes it’s even impossible. Now at Iso, our drug designers are, you know, literally sitting with their laptop browser-based interface being able to understand, make changes to their designs and see the impact of that. Stephanie Zhan: Incredible. So there are a couple interactions that AlphaFold 3 is focused on: proteins and nucleic acids, proteins and ligands, and antibody to antigen. Can you give us some good examples of the impact that AlphaFold 3 now has on the interaction of these different types of proteins and molecules? Max Jaderberg: Yeah, so protein and ligands, that’s the same as proteins and small molecules, so those two terms, ligands and small molecules, are synonymous. That allows us to understand how small molecule drugs interact. Then we can think about protein interactions. There’s a whole class of therapeutics called ‘biologics.’ —these are things like antibodies—that allow us to understand how they might interact with our targets. Opens up new modalities, and that also encapsulates the antibody-antigen interface. Stephanie Zhan: Yeah. Max Jaderberg: So if you’re designing an antibody, you want to understand how your antibody design is going to interact with the protein surface there. Stephanie Zhan: Yeah. Max Jaderberg: So it’s the same model that we can use across all of these different applications. Stephanie Zhan: What are the nuances of training a model like AlphaFold 3, and what are the benefits of using a diffusion-based architecture? Max Jaderberg: Yeah, it’s a great question. There were a lot of challenges we had to overcome to get AlphaFold 3 to work. One of the most interesting things was actually just how do we take something like AlphaFold, which was only working with proteins, and then input these new modalities, these new data types of RNA, DNA, small molecules. So we had to work out how to tokenize not just proteins, which we kind of knew how to do, but how to tokenize then DNA, how to tokenize small molecules. For things like DNA and RNA, that’s a little bit more obvious. We could tokenize in the bases, but then for small molecules we would really go to—we tried a whole bunch of different stuff, and really ended up that this atomic resolution tokenization. Worked super well. And then you have the question of okay, how do you actually predict the structure of this mixture of different molecule types? Stephanie Zhan: Yeah. Max Jaderberg: You couldn’t use the same framework as AlphaFold 2. And this is where diffusion modeling just really shone. Here we could actually model every single individual atom and the coordinates of every atom individually, and have a diffusion model be producing those 3D coordinates. And the tokenization that we talked about is conditioning the inference of that diffusion process. Stephanie Zhan: So interesting. Max Jaderberg: And this was a huge breakthrough. So, you know, we’re talking about on our leaderboard, just a massive step change, particularly in small molecule protein interaction accuracy. It was a massive step change, and something that really unblocked the rest of the project. Stephanie Zhan: Wow. So data, compute and algorithms, we know those three are important in all other adjacent fields, but I was surprised to read an interview with Demis where he shared that we’re not data constrained in biology. Can you share your point of view on that? Max Jaderberg: You know, I think it doesn’t matter what field of machine learning you’re in, you’re going to feel some data constraint. Stephanie Zhan: [laughs] Max Jaderberg: And I think the point here from Demis is that it’s not a real bottleneck, as in we can make progress with the data that is out there, that the data we can generate, and real progress can be made. It’s not oh, we’ve got to sit and wait 50 years for the world to generate data before we can actually make impact here. No, we’re not seeing that at all. Stephanie Zhan: Yeah. Max Jaderberg: There are modeling spaces where the data has been sitting around for years. Stephanie Zhan: Yeah. Max Jaderberg: That we can see, that we can make, you know, really substantial progress beyond anything that people have experienced before. Stephanie Zhan: Yeah. Max Jaderberg: Now does that mean there’s no opportunity for data biology? Absolutely not. Like, it’s going to be a fundamental part of how we continue to develop these models and these systems will be what data we go out and generate. And there, I think, there’s just a massive opportunity. In my mind, the data for machine learning in biology hasn’t actually been created yet. Stephanie Zhan: Yeah. Max Jaderberg: Yes, there’s a lot of historical data, but there’s a huge—but that historical data hasn’t been created for the purposes of machine learning. And so when you’re going out and thinking, “How do I create data to actually train my model?” you’re thinking in a very different way to how people have gone out and generated data in the past. And that, there’s a big opportunity there to explore. What kinds of data are missing?Stephanie Zhan: What kind of data do you think we’re missing here right now? And do you think that we need anything in synthetic data? Max Jaderberg: Yes. So I’m a massive fan of synthetic data. Actually, I have been for, since the very beginning of my career where, you know, I was generating synthetic text data just to overcome the fact that, you know, I was a PhD student with access to a couple of thousand images, and Google had millions and millions of images. And so instead I just generated tons and tons of synthetic data, and that unblocked things. And, you know, we’re seeing the same thing in the, especially the chemistry space, where we have good theory. We actually, you know, know a lot about physics. We have the theory of quantum chemistry and quantum mechanics, and we can create simulators out of that. We can approximate that and create more scalable molecular dynamic simulations. This gives the basis for a whole host of synthetic data. Then we have the models themselves that—especially we have generative models, this can actually generate data that we can use scoring systems to help really enhance the information content of this data. Stephanie Zhan: Yeah. Max Jaderberg: But I think one of the big open spaces will be on what’s called “in vivo data.” So data that you would normally measure on a real animal, something like a mouse or a rat. There’s some historical data on that, but you can’t generate tons of that. You can’t really generate any at all. Stephanie Zhan: Yeah. Max Jaderberg: Right? So then there’s a big opportunity to look to new data generating technologies. There are some incredible people doing things like organoids on a chip, so ways of starting to measure things that you would normally measure on a real animal but, you know, completely on a chip. So, you know, I think … Stephanie Zhan: So interesting. [laughs] Max Jaderberg: Yeah, there’s going to be a whole host of, like, new breakthroughs in data generating technology in biology and chemistry. That’s going to, you know, have big impact on how we think about modeling that world as well. Stephanie Zhan: Are you working on any of that internally, or are you hoping that other players fill in some of that gap? Max Jaderberg: So internally, we actually don’t have any of our own labs in Isomorphic Labs, but we work with a whole bunch of other companies. Stephanie Zhan: Yeah. Max Jaderberg: We generate a lot of data ourselves. Stephanie Zhan: Yeah. Max Jaderberg: A lot of proprietary data. We’ve seen amazing impact of that. Limiting factors in drug developmentStephanie Zhan: Makes a lot of sense. So there’s a point of view that modeling structure of molecules and modeling their function and the modulation function is very important, but not necessarily always the limiting factor in drug development. What’s your point of view on that? Max Jaderberg: Yeah, as I touched on one before, drug design is really, really complex. And that’s before you even get to drug development, which is where you take those designs and you start putting them into real people, clinical trials. There are so many bottlenecks throughout this whole design and development space. Drug development is, you know, how do we start to approach clinical trials? How should we test these drugs out in people? Stephanie Zhan: Yeah. Max Jaderberg: How can we do this in a really timely manner, but still a really safe manner? There’s a lot of bottlenecks there. Stephanie Zhan: Yeah. Max Jaderberg: That I think the industry as a whole, we will need to work out how to innovate in that space, especially as our predictive models of how these molecules will interact with people, how toxic they will be, as these predictive models get better and better, we will have to change the way that we approach clinical trials to really make use of that, ultimately to get therapeutics into the hands of patients who really desperately need them. Stephanie Zhan: Yeah. Max Jaderberg: Even in the design of molecules themselves, as we talked about before, it’s not just understanding the structure of these molecules. Stephanie Zhan: Yeah. Max Jaderberg: It’s not even just understanding how these molecules change the function of these proteins. But we need to understand how these molecules change the function of pretty much every single protein in our body. Stephanie Zhan: Right. Max Jaderberg: Because if we take this as a pill, it’s going to go everywhere. Stephanie Zhan: Everywhere. [laughs] Max Jaderberg: And that’s the major cause of toxicity, is when yes, you’ve designed this amazing molecule that, like, perfectly modulates your specific target that you know is key to your disease. Stephanie Zhan: But also affects other things. Max Jaderberg: But it also affects other things. Now of course, you do a lot of screening to protect against that, but the more we can predict that, the better. What’s really exciting from my perspective is if we’re creating these general models that understand how this molecule interacts with this target, but also any other target, then why can’t we just use that same model to understand how these molecules interact with the rest of our body? How do drug designers use AlphaFold 3?Stephanie Zhan: Right. So interesting. So what is now possible with AlphaFold 3 for drug designers? How are you using it internally? Max Jaderberg: So AlphaFold 3 gives our drug designers the ability to understand how their molecule designs really interact with this protein target—and this is the target disease. And so our drug designers can make changes to the design, and then see instantly how that changes the way that this molecule physically interacts with the protein target. Stephanie Zhan: Mm-hmm. Max Jaderberg: And that’s really, really powerful. Before AlphaFold 3, you would be completely blind to this. You wouldn’t actually probably know how your molecule is interacting with your protein. You’d be using your best intuition. Maybe somewhere down the line in the drug design project, you would get your structure crystallized with a particular design. That means going out to a real lab six months later, if you’re lucky, getting a resolved 3D structure. But even then, that’s just the 3D structure of a single design, not every single change that you make. Stephanie Zhan: Yeah. Max Jaderberg: So AlphaFold 3 completely changes the way chemists can do this design work. But I would stress that that’s nowhere near as far as we want to go. Because it’s not just about what these molecules look like in terms of interacting, we actually want to know how strongly these molecules interact with this protein. Stephanie Zhan: Yeah. Max Jaderberg: We want to know other properties of these molecules. We want to understand the way that these molecules interact with this protein, and how that changes the fold or the conformation of the protein, how that changes the function of the protein, how it might actually change the dynamics of the cell. There are so many questions, and these are these other AlphaFold-like breakthroughs that we’re working on that also go—you know, we have created incredible models for, that our chemists are using in this design process. Stephanie Zhan: Interesting. So you’re designing some drugs internally. What targets and programs are you focused on? Max Jaderberg: So we have a really exciting internal program of drug design projects. These are focused on immunology and oncology. We’ve been making some incredible progress there, and it’s been really exciting to see. Especially how these models have transformed the way that we’re actually approaching drug design on these programs. Stephanie Zhan: You’re also working with Eli Lilly and Novartis, and recently you announced an expansion with Novartis’s partnership. Can you share a little bit about what these partnerships look like? Max Jaderberg: Yes. So we signed these initial partnerships, two partnerships, one with Eli Lilly, one with Novartis in January last year, January 2024. And, you know, that was fantastic. They brought some really, really challenging problems to us. I think it’s no secret that, you know, the sort of targets that, for example, Novartis brought to us, these are sort of targets that, you know, the field and Novartis, for example, have been working on for, you know, 10 years plus. Stephanie Zhan: Wow! Max Jaderberg: So these aren’t sort of, “Oh, we’ll try things out” problems. These are for real, you know, hard things. Last year was an amazing year, both for our internal projects, but also for these partner projects to really see how well these models are working. It’s allowed us to really uncover new chemical matter, working out new ways to modulate these targets that people have worked on for a long time. It’s been amazing to see this new deal which has expanded on the Novartis collaboration, which I think is a real testament to some of the success of the early days of these partnerships. Stephanie Zhan: Congratulations. I think it’s an incredible milestone, especially just one year in. Max Jaderberg: Yeah. Building an interdisciplinary teamStephanie Zhan: So I’d love to talk a little bit about the team. You’ve built a truly excellent team composed of the highest caliber talent across many different fields—AI, chemistry, biology. And you’ve also brought outsiders into the field to help question traditional thinking. Can you share a little bit about how you thought about this? Max Jaderberg: Yeah. So the space of AI for drug design hasn’t really existed for very long, so the chances of finding a world expert at drug design who’s also a world expert at machine learning or deep learning is basically … Stephanie Zhan: Zero. [laughs] Max Jaderberg: Zero. Stephanie Zhan: Yeah. Max Jaderberg: Just because these fields haven’t coexisted for long enough. I genuinely think about a new sort of field of science that Iso is breeding because we are—you know, we have these people who really live and breathe the intersection of this. Stephanie Zhan: Yeah. Max Jaderberg: So, you know, but because we can’t hire these people, you know, I really think about how do we bring the world experts at drug design and medicinal chemistry and the world experts at machine learning and deep learning, and get these incredible people sitting side by side. Because it’s not just enough to have these amazing people sitting in their isolated teams. Stephanie Zhan: Yeah. Max Jaderberg: We need people sitting side by side, speaking each other’s languages. Stephanie Zhan: Yeah. Max Jaderberg: With a lot of empathy, a lot of curiosity. Curiosity to understand this new science, to really build intuitions in your own language. And we’ve seen just such amazing things come out of this dynamic way. Stephanie Zhan: Yeah. Max Jaderberg: You really have, you know, a generalist machine learner who doesn’t know anything about chemistry or biology … Stephanie Zhan: Yeah. Max Jaderberg: … start to come in and understand the problems of a medicinal chemist and a drug designer. And when I think about even hiring machine learners and machine learning scientists and engineers for the research that we’re doing, I’d say, you know, 60, 70, 80 percent of the people on our team have no prior knowledge of chemistry or biology. Maybe, you know, high school or university level. And that can actually be a real asset. Stephanie Zhan: Yeah. Max Jaderberg: Because you come in sort of a little bit naïve. Stephanie Zhan: Yeah. Max Jaderberg: And as long as you’re curious, I think, one of the key things is asking, you know, the curious questions, asking the, like, stupid questions, and then that allows us to come at the problems from first principles. Stephanie Zhan: Yeah. Max Jaderberg: It almost allows us to break through the dogma of previous experience, and how people traditionally approach these problems. We can think ground up from scratch. And that’s a lot of the mentality of how we think about creating these research breakthroughs. Stephanie Zhan: A little naïve and highly curious and high agency is a very good thing. [laughs] Max Jaderberg: Yes, exactly. Exactly. AlphaFold ServerStephanie Zhan: So in November last year, you also made a very big move in launching the AlphaFold Server, which releases code and model weights for academic use. Can you share a little bit about why? Max Jaderberg: Yes. So I mean, AlphaFold has a long, long lineage of being open for this academic and scientific use. And it was really important with this latest breakthrough of AlphaFold 3 that we make sure that this scientific community has access to this functionality. Because yes, AlphaFold 3 is going to be incredibly useful for drug design—it already is. But it’s also useful for many other areas of fundamental biology and just understanding biology. And people are using these; people are using our AlphaFold 3 Server and model it in very, very creative ways, so it’s very important for us to make sure that there is that free use for non-commercial academic work. And it’s been incredible to see the take up of that and the use of the server. Stephanie Zhan: I’d love to talk a little bit about the future. Can you give us a tease of what else is to come with AlphaFold? Max Jaderberg: In terms of structure prediction as a problem, in my mind I want to completely solve this. I think AlphaFold 3 is a fantastic step on the way of that, it’s a significant breakthrough. But, you know, it’s not 100 percent accuracy. Stephanie Zhan: Yeah. Max Jaderberg: What does even 100 percent accuracy mean in this space? Like with a lot of areas of science, as you start to push the boundaries, you see that the problem opens up into even more problems. You know that’s the addictive part of doing science, right? And I think that, you know, AlphaFold 3 is a good example of that, where as you start to get these capabilities, you see that actually the even more deeper problems that we want to be working on and stepping towards. So yes, understanding structure better and better and more accurately is always going to be interesting for us. But then it’s also not just necessarily about static structure. So AlphaFold 3, it models these crystal structures which are almost static, crystallized versions of how these molecules interact, but in reality we don’t have crystals inside of us. These molecules are in solution, they’re moving about the dynamic. So you can think, “Okay, well maybe understanding the dynamics of these systems is actually also going to be really interesting.” The GPT-3 moment in AI biologyStephanie Zhan: Yeah. What does a GPT-3 moment look like in AI biology? And when do we get there? Max Jaderberg: So if I think about GPT-3, this is really a generative model, so something that’s generating text. And the GPT-3 moment for me was, you know, crossing over that boundary between yeah, we’ve got generative models of text and they generate some stuff and it looks like text, but I’m not convinced that it’s generated by a human. Stephanie Zhan: Yeah. Max Jaderberg: And GPT-3 started to be that first point where you’re like, “Oh, shit!” Stephanie Zhan: “Oh, shit!” [laughs] Max Jaderberg: This is like—this kind of looks like a human. And so this generative model is actually recreating the distribution of data that is trained on. And what is a generative model? A generative model is something that fits the manifold of data that it’s trained on. So when I think about this applied to biology, you can think about these generative models actually starting to recreate—at that GPT-3 moment, recreate what things would actually look like in reality. And that’s quite exciting because that means that these models are spitting out things that either they actually exist in the world. Stephanie Zhan: Yeah. Max Jaderberg: And we can kind of validate that, or maybe even discover new things that exist in the world. Or they could exist in the world. Stephanie Zhan: Yeah. Max Jaderberg: Which means that they could be things that we could design or manufacture or create that would actually be stable and work and exist in our physical reality. Stephanie Zhan: Yeah. Max Jaderberg: And I think the cool thing about this in biology is that, unlike with language, where with language, when it generates something at human-level quality, we can understand that because it is human derived. But a lot of problems in chemistry and biology, we even struggle to understand ourselves. And so when we get to that GPT-3 moment, I think it will look a lot less like GPT-3, but feel a lot more like move 37 in AlphaGo. Stephanie Zhan: Hmm. Interesting! Max Jaderberg: Where we’re starting to see things that are beyond human understanding, but that do exist in the real world, that exist in our physical reality, but are beyond sort of human comprehension. Stephanie Zhan: Right. Max Jaderberg: And that’s just going to be mind blowing. In fact, you know, we’re starting to see that internally with our generative models, that we’re creating designs that a human drug designer would say, “Hmm. I’m not so sure about that. I much prefer this.” And then you test it out in physical reality, and the generative model is correct and the human is wrong. Stephanie Zhan: That’s fascinating. I love the move 37 analogy. When the model starts to see elements of creativity and surpass the human. Max Jaderberg: Move 37 was this amazing move during the AlphaGo games against Lee Sedol. It was, you know, the 37th move of the game, and it stunned the world, stunned the Go world because it was uninterpretable by a human. Stephanie Zhan: Yeah. Max Jaderberg: It looked like a mistake. No one had ever played this move in the entirety of, you know, thousands of years of human history playing Go. And it turned out as you unrolled the game, that this was the critical move that allowed AlphaGo to beat Lee Sedol in that match. Stephanie Zhan: Yeah. Max Jaderberg: And we’re going to see so much of that sort of behavior coming out of these models. Stephanie Zhan: Yeah. Max Jaderberg: Especially when we’re applying them to things outside of native human understanding, like chemistry and biology. When will we see the first AI-generated drug in clinic?Stephanie Zhan: Yeah, I love that. Also our punchline today. [laughs] So when will we see our first AI-generated drug in clinic, and also in phase one, two and three trials? Max Jaderberg: So we’re making amazing progress on our drug design programs. And, you know, the thing I think about actually is as we start to get a whole bunch of these AI-designed assets, these molecules, getting to clinical phase, how can we actually start to think about engaging in that clinical development to get these molecules to people as fast and as safely as possible because there’s so much unmet medical need? So yeah, here I think about, you know, what are going to be new ways to engage with regulatory bodies, what are going to be new ways to incorporate our predictive models for not only how this molecule works for the disease, but how—as we talked about, how it interacts with the rest of the body, you know, the types of toxicity it may induce. I think there’ll be a lot of opportunities to think about just streamlining and speeding up this process. Stephanie Zhan: Yeah. Max Jaderberg: Maybe even completely changing the way we think about human clinical trials as our AI models become so—we can design these molecules so much quicker in a much more targeted manner with so much more knowledge about how they work. Stephanie Zhan: Yeah. Max Jaderberg: So that’ll change the game. But I think we’ve got a long way to go as an industry to really work out how that changes. Stephanie Zhan: Yeah. Last question. As Isomorphic succeeds, and potentially as a whole field succeeds, what happens to the traditional world of pharma? Max Jaderberg: I think they become—you know, in some sense, pharma will be using AI. I think there’s no world where in five years time you will be designing a drug without AI. Like, that is an inevitability. It’ll be like trying to do science without using maths. Stephanie Zhan: [laughs] Max Jaderberg: AI will be this fundamental tool for biology and chemistry—it already is, at least in Isomorphic’s world—that everyone will be using. So it’s not going to be, “Oh, is it pharma or is it AI?” It’s going to be one and the same in the sense that the whole industry will adapt to that. Stephanie Zhan: Yeah. Amazing. Max, thank you so much for joining us today. This was a fascinating conversation. Max Jaderberg: Yeah, it’s been a pleasure. Thank you. Mentioned in this episodeMentioned in this episode:  Playing Atari with Deep Reinforcement Learning: Seminal 2013 paper on Reinforcement Learning  Capture the Flag: 2019 DeepMind paper on the emergence of cooperative agents AlphaStar: 2019 DeepMind paper on attaining grandmaster level in StarCraft II using multi-agent RL AlphaFold Server: Web interface for AlphaFold 3 model for non-commercial academic use <div class="framer-mcvrsd" id="