人类点击数据是个 bug：前推特 CEO 要给 AI agent 重造一个搜索网络

id="podcast-transcript" data-framer-component-type="RichTextContainer" style="transform:none">IntroductionParag Agrawal: Our view at Parallel is that human click data is a bug. An agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information, and we can benefit from a lot of the research that has gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments. And that’s the arguments we made back then, that actually now it’s way more tractable as a problem because of the existence of agents, not just as a technology, but as a distinct customer. Main conversationSonya Huang: Parag, thank you so much for joining us today. We’re delighted to have you on the show. For those who don’t know, Parag of Twitter CEO fame, was the CEO of Twitter before selling it to Elon and is now back on the founder arc. You founded a company called Parallel Web Systems, which is scaling up agentic search for the agentic web. We’re very excited to have you here today to talk about the future of search and the future of the internet. So let’s get started. What’s Parallel? Parag Agrawal: Thank you, Sonya, for having me. Thanks, Andrew, for joining us. I’m really excited about this conversation. At Parallel, we’re building a bunch of technology in order to allow agents to search and use the web. So just like humans forever have figured out how to use browsers and search engines and clicking and browsing around the web to access information, agents need to do the same things. We started Parallel with the bet that agents would do it 1,000x more than humans ever have. And as a result, we need to reinvent the technology that can power search for agents, but also the business models that go alongside it. And that’s what we are doing at Parallel. Sonya Huang: Okay. I want to go back into what makes human and agent search so different. But before we get there, you’ve told us that you’re unlearning a lot of the lessons that you learned from running Twitter as you build Parallel. Why is that? Parag Agrawal: Listen, when I was at Twitter in leadership roles, Twitter was a post-product-market fit, extraordinarily scaled business where your feedback loops were from the hundreds of millions of customers using the product for 30+ minutes every day, right? In that world, you operate differently than a pre-product-market-fit company based on the premise that in a few years a new customer is going to show up on the internet, and let’s go build technology for the not-yet-here customer that we are all learning every day and every week. Sonya Huang: Fantastic. Okay, let’s talk about Parallel. Andrew Reed: I have a question actually. Can I jump in? Sonya Huang: No, you cannot, Andrew. Andrew Reed: I’m Andrew. I didn’t get an introduction, but I’m also happy to be an inaugural guest on the Training Data podcast. Sonya Huang: Andrew is making his podcast debut on Training Data. We are honored to have you finally here. Andrew Reed: Thank you. I feel like all of us in Silicon Valley have a very surface-level idea of search. What is search? The job to be done is getting the answer; makes sense. But we all know that there’s crawling and there’s an index and there’s ranking. But maybe let’s take a big step back and explain what is the problem of web search, be it for a human or an agent, and then we can dive into the differences. Parag Agrawal: The problem of web search—and we all know it and experience it— when we want to find something and we do not know where it is on the web, we go to a search engine like Google, and the search engine then hopefully surfaces the answer to us in the most convenient of locations. That’s the base problem. Now in order to do this, what is the search engine doing? The search engine is going and crawling the web, which is finding every URL that’s out there, trying to read it, trying to organize that information in what might be called an index, so that by having all of this stuff in memory in one location, you don’t have to—once the query comes in, you don’t have to spend time loading up pages because you already have them. You’ve already done preprocessing to organize them. And then when the query does come in, you quickly understand the query, you find the most relevant results. And then there are many, many—because you’re essentially taking hundreds of billions of pages and narrowing it down to what, five or ten, or ideally one, in terms of what someone is looking for right now. So you go through these multiple stages of retrieval and ranking in order to surface the most relevant result. So that’s the broad problem. One way to think about it is it’s a billion-to-billion matching problem, right? So you have hundreds of billions of pages and hundreds of billions of queries over time, and you need to figure out how to matchmake across these two. Sonya Huang: That sounds like an enormously expensive infrastructure challenge. And I think for the longest time, basically only Google and Bing had done full web-scale crawls and indices. Why did you think that you as a young company could go off and tackle that problem? It seems like a problem of the giants. Parag Agrawal: So it is expensive in the long term. So what’s actually interesting is that when I was starting the company three years ago, you could imagine a world where one, some of the reasons it was difficult for others to compete, which was not having access to great feedback, in terms of is this a better search result than this? To collect that feedback at scale, there was a problem around human ratings. There was a problem around click data. You need to access those. Now it is, of course, expensive to crawl the entire web and index it. But as you think about what agents and the large models that we now have access to have enabled, it’s the ratings data can be now created by experts way more cheaply. Our view at Parallel is that human click data is a bug. An agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information, and we can benefit from a lot of the research that has gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments—and that’s the arguments we made back then—that actually now it’s way more tractable as a problem because of the existence of agents, not just as a technology, but as a distinct customer. And then we figured out a way to go about building this business which did not require us to go spend all of the money on infrastructure upfront before we could service a customer, right? So if you can paint a path of incrementally being able to build an increasingly larger and more sophisticated index over time as you solve problems for more and more customers, that’s the insight that actually convinced me that this was a viable problem for us to take on. Sonya Huang: And how do you make that happen? Because I imagine this is one of those things where customers want full coverage day one. So how do you go about making that happen? Parag Agrawal: When we first launched the product, we did not launch a search product first. We launched a search agent product first. Our search agent could go essentially crawl the web after a query arrived. So if you’re doing deep research, you have patience to the extent of a minute. And we’ve had products which sometimes take 10 minutes of research. That’s a lot of time to be able to crawl a lot of pages, if only you have enough of a map to know what to prioritize crawling, right? So you can make up for shortcomings. Like, index is oftentimes—you can think of it as a latency optimization. So if you give up on that dimension, if you’re competing with humans—that’s why our search agents were competing with the alternative, being outsourcing to humans to curate amazing data, right? So we said it seems like humans sitting on search engines are way easier to compete with than a search engine on day zero. So by building a product that was a search agent to do real work on top of web data, we were able to incrementally go build our index. Sonya Huang: What are some examples of what people built with your search agents? Parag Agrawal: In the very early days, there were search agents being built with us for doing some kinds of insurance underwriting workflows and claims processing workflows. People in sales were doing all kinds of sales data enrichment. People in finance who would previously and historically go to build a model, collect data by sending it overnight to a set of humans who would curate data that would feed into a modeling exercise they would do, would start using us to do that instead. And so we were going initially after replacing where there was outsourced human work on top of web data, in order essentially to collect evals, to run agents, to figure out what search for agents should look like in the first place with empirical use cases instead of theoretical evals. Sonya Huang: I see. You’re trading off the crawl for inference time compute effectively. Parag Agrawal: Yeah. Andrew Reed: You guys were started before the term “neo-lab” came into existence, and you have a web systems product, but you also have a bunch of AI people. From the outside, before we met for the first time, it wasn’t obvious to me how much of a web systems company or an AI company that you’ve built. The more time we spent together, the more clear it became that by some definitions, this is a neo-lab. Do you wanna talk about where the research side of what you’re doing comes in? Not just the infrastructure and system side of what you’re doing. Parag Agrawal: So we don’t categorize ourselves as a neo-lab. Andrew Reed: Well, of course you would not. Parag Agrawal: No, because I don’t think our output is a model. Maybe my definition is broken. My definition of a neo-lab is an output is a model. I think our output is a complement to a model. What we build is something that multiplies on top of a model in order to give—either you can call it the model gets better, or the agent built with the model gets better and has superpowers, right? So we always want to be in a place where whenever someone ships a better model—a neo-lab, for example—if somebody else ships a better model, they have a higher hill to climb. For us, somebody ships a better model, they have now unlocked four more use cases where we can be valuable. Now whether we need to do work that might be framed as research, that remains. So in that sense, we do have to do research, but we’re also not pre-training extraordinarily large models. In fact, our job is to compress things down into tiny ranking models, right? I think if you go back to my framing around this 100 billion to 100 billion matching problem, every query is essentially: Give me a thousand tokens from a trillion web pages on the web, and make sure they’re the right thousand tokens. That’s the query that we are getting effectively to our search engine. And so now what you have to do—and until last week we would give ourselves three seconds to throw a bunch of compute at read time to do it. Last week, we shipped a product which now does it in 200 milliseconds, right? So you have now that much time to go figure out how to pick those. We’re trying to organize information in memory across the memory hierarchy in a way that we can access it fast. We’re trying to figure out how to train what model to select the best thousand tokens. And so there’s a large amount of research that goes into figuring out how to take a pre-trained model of some kind, adapt its architecture for this new problem, and allocate effectively compute in a way that produces the best output with a limited compute/latency budget. Sonya Huang: So a lot of these use cases seem like the deep research kind of shaped use case. And when I think deep research, at least in the earlier incarnations, it was effectively like an agentic loop with the model reasoning and then basically just calling a search engine, calling Google or some proxy to Google Parag Agrawal: Yeah. Sonya Huang: Right? Why is that insufficient in your eyes, versus what I’m hearing from you is agentic search is kind of a net new capability? Parag Agrawal: So calling Google for every query in a deeply searched thing, it gets you somewhere, right? If you use Parallel Search, you will, for the most part, use under half the tokens in your agent. It will become more accurate and to be faster end-to-end. And so you can think of it as—but every time, if you use only half the tokens, if your model is context limited or memory limited, you can now do more problems. You can do the same problems cheaper or faster. So everything to me, when you have an infinite appetite for information and relevant information for all kinds of work, it’s at its core an optimization problem around quality, cost and latency. Every model advancement is about how do you squeeze out more intelligence, and then how do you distill it down into keep most of it at a tenth of the cost, right? And that applies to search as well. And every time you can produce all of the signal with less noise in your search results to give to a model, you now give the model the ability to do more. And so a lot of the bet here is intelligent compute allocation across the model layer and the agent layer down to the search layer. The other interesting thing to observe is the interface actually changes when you think about serving agents versus humans. Humans rely on keyword search. With agents, we’ve had to innovate quite a lot on what does an agent tell our search engine? The more we know, the better we can do. Sonya Huang: Interesting. So humans can only hold a few words in memory, basically, versus an agent? Parag Agrawal: No, I think we can. We’re just lazy. Sonya Huang: Yeah. Parag Agrawal: Right? We can have a conversation, we can hold a lot of words. But when you start typing, we want to tell Google two incomplete words with a typo in there and hope for the best, or rely on some form of a dropdown autocomplete thing to avoid typing those three words in the first place. So we’re fundamentally lazy. Turns out, like, models are less so. Fewer typos, better specified queries, perhaps longer queries, less for the search engine to guess what the agent might want. So you get to solve a different class of problems. Andrew Reed: One other thing about humans is I feel like we know how to decipher the pre-AI slop that populates many of the top-ranking Google results for best product for X, Y, and Z, and you get all the affiliate advertising things. And yet obviously they rank highly. Sonya Huang: I’m very susceptible to it. Andrew Reed: Are you? Yeah, I feel like I have this incredible—I can just see it coming a mile away. How do agents deal with things like that, right? Where these are well-trafficked pages, they have seemingly good answers, but you just know they’re not real. Parag Agrawal: Perhaps with Parallel agents, you have to deal a little bit less with that. Let me tell you why those pages exist in the first place, right? Let’s work through a simple example. If you ask for a public company’s most recent financials, like just the headline revenue number, we can sit here and know that there exists an authoritative filing with the SEC which has that number—perhaps on page 73 of a PDF, but that is the authoritative number, right? Now imagine Google decided, like, I care about authority. And whenever you ask that query, that’s the first result you see, right? And then you click that. This PDF takes, what, like 3.5 seconds to load instead of the second that another page does. You’re already frustrated. And then you see an earnings page. Andrew Reed: 16 Alpha. Parag Agrawal: That’s the alternative. But here you have to now grep your way to page 77 to find your answer. We’re lazy. We’re not going to do the work, right? So now there exists an entire class of content on the web, which is like okay, this information is needed by a lot of people. It is worth putting it on a page that loads fast, where this information is above the fold. All of you can go there. It’s 99.99 percent right, so you’re not so skeptical when you go there that this is 100 percent wrong. And it’s added real value in the process because it identified out of the 300-page earnings report, 20 bits of information that should be above the fold. Andrew Reed: Yeah. Parag Agrawal: Right? And so you can call it slop, pre-AI human slop, or you can call it catering to a lazy human and being successful at SEO. The good news is with agents, we’re not making the agent click around and fumble around and grab a PDF, right? We’re taking an excerpt from the most authoritative place on the web and trying to bring it to the agent’s context window. And so we aren’t forced into this weird trade-off. And this trade-off existed with humans in the first place, because with standard browsers and protocols and everything, we just didn’t figure out how to have us point exactly consistently across everywhere on the web to the exact right highlighted tiny paragraph, right? With agents, we get to bring that to the agent’s context and then let it figure out what’s next. Sonya Huang: Can you walk us through what actually happens when one of your developers sends a query off to your Agentic Search API? Parag Agrawal: It’s in some sense pretty standard. So we run some models to figure out what this query is and enrich it to figure out how it will flow into the rest of our system. We have a bunch of indexes which organize different subsets of the web in different ways. And so the first layer will essentially craft queries for each of these different systems. Each one of these systems perhaps is our sort of big index. One of these is perhaps our fresh index. One of these is perhaps—some people will describe it as like a knowledge graph. Some people will describe it like a structured index. There are a bunch of these, right? So you’re now deciding which ones this query needs to go to. You’re figuring out what is the query rewrite for each of these. Then each of these has a big retrieval layer and then a ranking layer and then more ranking layers. So you’re going to try to boil down tens, hundreds of billions of URLs or documents down into thousands, tens of thousands, down into specific excerpts and paragraphs in those tens of thousands with more and more bigger models running at each stage with different architectures, pulling more features to ultimately get down to here are the thousand tokens I want to bring back to this AI which has the highest signal, right? And now if you look at our various versions of our search API, they just throw for different latency and cost constraints, different amounts of compute at various points in this journey to hit those limits. So in the abstract, it’s simple, right? What’s interesting is what each of these models that I described, how you curate and collect the training data for them, right? How you build those models and optimize them. The index itself, how you use the memory hierarchy to store it, to be able to hit certain cost, quality, latency thresholds. Sonya Huang: What’s the north star from a quality perspective? Like, you know in Google, there’s that did you get the result you wanted in the first three answers or something? What is the equivalent north star for you? Parag Agrawal: So I think of—I don’t know if I’m right on this, but my take is that a billion-to-billion matching problem is a forever problem. And so the real question is at what point incremental optimization isn’t worth the squeeze, right? So I don’t think there is a thing as, like, okay, we’re done on improving this thing. The question is at some point it’s going to get harder and harder to improve this thing, and it just won’t be worth it. But I’m hoping that we don’t get there actually, because if you think of what we’re doing with AI, we will have more intelligence that gets cheaper every few months. And as a result of it, it will come down to having great models which are cheap, having great information, matchmaking across a need and all of the information available to you, whether it’s your own or on the web, and doing something unique and differentiated with it to produce more knowledge, right? And anytime you can do something 20 percent better than somebody else, that might give you an edge. So why wouldn’t you? So in the super AGI-built worldview, it feels like if you can push on quality across web search and in the model layer, why wouldn’t you? Sonya Huang: I’ve heard a point of view that this is so fundamental to the model companies that they’re just going to own it. And in part because as they’re collecting data for pre-training, that’s a very, very expensive infrastructure exercise. That kind of is your source of truth for the crawl. What do you think of that? Parag Agrawal: I don’t see empiric data on the ground to support that view. To build a fresh web index, I don’t think that crawl is particularly useful. And let me frame why. So if you think of what we are building, we’re building a complement to models. We like to crawl things that people don’t like to crawl for pre-training. Because if the model already was trained on it, it’s not useful. Model companies, for the training, aren’t patient enough to go in a completionist way, try to wait for really slow random JavaScript to load, because the number of tokens you get per amount of compute you throw at it is like one order of magnitude, two orders of magnitude. Too bad. And so is it worth the extra effort to get these tokens? For us it is because we’re completionists, right? For them it’s like, I’ll take X trillion tokens. So that’s one. Now I do think this is a core part of every agent, right? My worldview is that if you’re buying LLMs for doing work, for nine out of ten use cases, you will want them to have access to the web and create search infra optimizing for agents. So it is a real adjacency for all kinds of LLM inference. And that supports your view that model companies could, should have the best-in-class infra for it. So now they can build it or they can buy it. And that’s the conversation. And we will see who builds and who buys and who partners and how things evolve. Sonya Huang: Are you partnering with any of the model companies that you can share? Parag Agrawal: We want to. We will. I can’t share anything on that. We did announce—and I don’t know how your definitions of model companies—we announced today, actually, that we are working with Google Cloud to be a search and grounding provider for their enterprise agent APIs. So if you think of grounding Gemini models or other models available on GCP, when you build agents on GCP or chat apps on GCP or do any other inference with LLMs on GCP, when you attach web search to it, your options are Google Search or Parallel Search. And Parallel Search is product integrated, the integration is optimized. We’ve spent time with technical teams and training teams and product teams and commercial teams to make sure that when people use Gemini models with Parallel, they get exceptional and great results. So yes, there are these partnerships now emerging. I bet that there’ll be several of these. They will all look somewhat unique. Sonya Huang: That’s a big deal from the search king. Congratulations. Andrew Reed: Google’s the original neo-lab. Parag Agrawal: I don’t know how to frame Google as a model lab versus a hyperscaler. And I don’t know what precise lessons to learn from this one, whether it applies to other labs or to hyperscalers. And so we will see. Andrew Reed: The way that I use voice agents—and I need to get reservations for dinner at night, what restaurant should I go to? And then that request gets fulfilled by the agent. When we talk to a lot of the Parallel customers, there’s this background agent, whether it’s the monitor product, these agents that sort of are always watching the world or watching the web, and when something happens, then they go off and take actions and do something with it. It might be worth—because if we think about what does 1,000x more mean, there’s the depth of research and then there’s just what is actually initiating the tasks. Is it a human initiating the search or is it the agent itself? You want to talk about that dynamic a little bit? Parag Agrawal: Yeah. So there’s a bunch of dimensions here. Let’s go back for a moment on search agents. If you run a typical search agent, even without doing deep research, it’ll do somewhere between five to twenty searches, even if it answers within a few seconds. Because why not, right? So already, if you transition from using ChatGPT Medium, or—I know all the categories, but somewhere not on the high tier—Instant, Low, Medium—every time you write a prompt to it in ChatGPT, it will do five to ten searches. As you dial it up, it’ll do hundreds and thousands of searches. So one interesting thing to observe is a human action to a multiplier on number of searches that happened, right? So just by using an AI app, you’re kind of multiplying your way to perhaps one order of magnitude more searches. Now a lot of our initial takes on the product and the market were to go after bigger multipliers than even that, right? So we were much more interested when you said I have a portfolio of 10,000 small businesses where I have given out credit to. For all of them, every month I have this human process that runs to feel out how my risk is going up or down. Now can we—and it relies on a bunch of web data. We’re trying to use agents for doing this. So here a developer is effectively—the multiplier there is hundreds of thousands or a million in terms of the number of web searches that happen because a human goes and programs that instead of now doing this process every month, we can do it every week, right? So doing a lot of searches when this agent runs every week to create a dashboard on the portfolio and a collection of action items that somebody needs to look at. Now you can go to another example, which is even more interesting. I don’t know, do you use something to do meeting prep documents for you all that’s an agent? Sonya Huang: There’s a Sequoia one. There’s a Sequoia agent. Andrew Reed: Yeah, yeah. I was going to say James Flynn. He’s one of our great young guys. Parag Agrawal: So I use Notion’s agent—and you can build custom agents which look into all the internal data that we have at Parallel, plus all of the web data using Parallel’s APIs to create meeting prep docs. One time I went and created one prompt to build this custom agent. Now it does tens and hundreds of web searches for every meeting I have. Every time I build a new agent for a new use case, that keeps multiplying. So I think the path to these background agents doing more and more and more work all the time for us, it’s only going to be bounded by value versus spend. Andrew Reed: Mm-hmm. Parag Agrawal: You know, I don’t think it’s rational right now. I’m probably spending more on it than I should be, but it’s not too much, so I don’t care yet. So there’ll be some rationalization in all of these agents. But I think we will deploy background agents to the extent that there is incremental value in doing that compute. And the same thing applies to searches. All of these background agents will do a bunch of searches. And so our first set of products were really obsessively focused on these, in part because we were building, growing the index, and we decided that our company was based on three dimensions: quality, cost, latency. And for the first couple of years, we said let’s focus, let’s ignore latency, and let’s just nail the other two, because optimizing systems distilling to smaller models is much more a known art than unknown research, right? So once we achieved the best quality search and search agent products at every price point, we’ve started working on latency. And that’s what we shipped with a product we call Turbo now. It is the fastest, highest quality agentic web search on the market by a lot. Sonya Huang: Do you think there are more agentic queries than human queries on the web now? Parag Agrawal: I don’t think yet. I don’t think we’ve reached that point. Sonya Huang: Just given some of the multipliers you mentioned and given the background agents, it seems ... Parag Agrawal: I think I don’t have to remind you, but we are early. We are very early in agent adoption. You go step outside of our bubble, people haven’t heard the word Fable. So we are very, very early. I think there are now people like me who are probably operating at the 1,000x. I don’t know, what do you think? How many Google searches a day did you do three years ago before ChatGPT? Andrew Reed: 20 or 30? Parag Agrawal: Yeah, I would have guessed something like 20, 30. I think today, if you just look across at all of my agents, I bet they’re doing 1,000x more than that. Maybe 100 to 1,000. If you count some of the things that happen at my company which isn’t assigned to a human, it might easily be more than 1,000x. But I think we are the outliers rather than the norm. So I think we are very, very early on this journey. I do think—I recently saw, I think it was Cloudflare that said that in their monitoring of web traffic, the AI traffic is about the same as human traffic in terms of page reads, which is slightly different from searches, because it includes perhaps all crawlers that are out there, and a bunch of other stuff. But I think it’s going to happen. Sonya Huang: So maybe this is a good segue to talk about a topic that I know you are passionate about, which is the economics of the internet as we know them. Some of the fundamental assumptions there, you know, human eyeballs, scarcity of attention, seems to be falling right in front of us right now. Are the economics of the internet broken now, and what’s going to happen? Parag Agrawal: Yeah. No, I think this was perhaps part of the thing that totally convinced me that this needed to be done by me and build Parallel. I know most people hate ads. I used to do ads and build systems for ads. Sonya Huang: I love ads. Parag Agrawal: Oh, wow! Sonya Huang: I love shopping Parag Agrawal: Okay. Yeah, so you get good ads. Sonya Huang: [laughs] Parag Agrawal: I don’t know if I love ads, but I intellectually love ads because ads are the reason that so much amazing content and technology is available for free to all of us, right? Like, Google Search wouldn’t be free without ads, Twitter wouldn’t be free without ads. And these are truly useful pieces of technology. A lot of content on the web out there that we can access for free wouldn’t be free if not for ads. And so ads are a very efficient monetization scheme. It’s a very efficient monetization scheme because it is exceptional at differential pricing, right? So most queries Google loses money on. Some of them make it up, and it’s an extraordinary business with extraordinary margins. Same with Twitter. Like, most users ... Sonya Huang: Like for shopping. Parag Agrawal: Exactly. Sonya Huang: You’re welcome. I’m subsidizing the free information you guys are getting. Parag Agrawal: You are subsidizing all of us. Thank you. But I think ads are extraordinarily efficient at differential pricing and monetization on the web, which is why it has been a dominant business model. Now the core assumptions, as you noted, going into it, around limited human attention to translate into outcomes, if humans don’t show up and agents show up on the web, what does this mean? How does the business work? And yeah, I think this was the—if we don’t figure out a new business model—we’re seeing it already, right? Like people are going to say, “Okay, I don’t want my content to be accessed by an agent, because I have a business model for humans. So I actually want to go pay someone to SEO optimize myself so more humans show up.” But then if their agent shows up, I’m going to cut it off, right? Which seems confusing and disconcerting. Ultimately this agent is acting on behalf of a human, but we haven’t found business model alignment. Sonya Huang: Yeah. Parag Agrawal: In a way ... Sonya Huang: We can’t monetize that visit. Parag Agrawal: You can’t monetize that visit. Sonya Huang: Yeah. Parag Agrawal: So let’s say you are in the business of subscriptions, right? So you get a thousand humans, and you convert 20 of them into monthly subscriptions. Right? You don’t know how that loop works yet with agents. You can’t distinguish, you don’t have statistics, you don’t know if these will lead to subscriptions or if they’ll just keep stealing your content as nameless agents. And so there are real challenges around the old business models breaking and us not figuring out real scalable new business models, right? So if you own high-quality content and you currently put it on the web, your one available business model pre-Parallel was to be in the head and be able to transact with a lab on some fixed fee contract. That’s literally—which includes some amount of training and liability and then inference time access, right? That was your one option. That option is not available to most content on the web. It’s a very head phenomenon. And even for the head, it is a broken business model, because when AIs or inference grows, let’s say 7x this year and another 7x the next year, on this 50x, their deal size is not growing 50x. None of them after signing a two-year deal believes that their share isn’t going to decline materially at renewal. And so these are fixed price constructs in the world of AI inference, which doesn’t drive for sustainability for all of these businesses. Now our solution is trying to learn all the lessons from my work at ads on having transacted—like, I sold Twitter data to OpenAI, having transacted on that side to figure out what actually might work and be incentive-aligned. Sonya Huang: So what might work? Parag Agrawal: Efficient differential pricing. Paying differentially a lot for extraordinarily high-value content. Paying a lot for high-value work accessing the same content. So it’s differential on both dimensions, quality and value of work being done with it. And a way of doing this scalably and not just with bespoke deals. So those are the properties needed for any reasonable solution. And the biggest property of it all is incentive alignment, right? Like, at what point do people want to collaborate into this enterprise? Sonya Huang: Why do the model companies need to pay anything at all? Parag Agrawal: You should ask them. My understanding is one, you want fresh data during inference time to be able to display it in products like ChatGPT or Claude. Two, you want training data. And three, you want some liability protection for training that you already did. And so the payments are some combination of these three things. And I can’t be sure of how they value each of these three. Sonya Huang: You said even a very simple query will go do 10 searches. And so how do you do attribution between all the different sources that boil down to one paragraph response? Parag Agrawal: Yeah, at Parallel, we like to build models. Sonya Huang: [laughs] I heard. Parag Agrawal: So no, I think it goes back to my point around incentive alignment. So before we try to build a model, let’s try to figure out if you were going to try to do this intellectually, theoretically, how would you go do it? So you have to ask the question of okay, how much incremental value did somebody’s content add? Right? So you can run all of these simulation exercises. You take one piece of content out of the corpus and then you say, “Let’s run the agent. Let’s see if the quality of our output declined, how much to claw back that quality? Perhaps if I threw a little bit more compute or a better model in some way, could I claw back that quality? Oh, I could. It cost me a cent. Huh!” I could get the quality I lost by not having this source. My alternative was to throw more compute, a cent worth of compute, to get that same quality. It might be a different result, but as far as the end customer value is concerned, it’s the same quality. And so you’re like, okay, this source is worth close to a cent, it feels like, right? That’s intuitive. Now the formalization of this kind of an intuition is the core framework we use. It’s called “Shapley values.” Sonya Huang: Yes, let’s go. What’s a Shapley value? Music to my ears. Parag Agrawal: It’s a game theoretic—you’re a game theorist? Sonya Huang: I did a bunch of game theory, yeah. Parag Agrawal: Yes. Shapley value is this very theoretical mathematical concept. It is used in SHAP values in feature importances for those who are ML folks here. Let’s say the three of us collaborate on something, and the whole is bigger than the sum of parts in that moment. But the theoretical question is okay, how do I divide up this sort of bigger pie that we created by collaboration so all three of us have incentive to collaborate, right? And Shapley values is a mathematical way of effectively answering this question. Now that sounds amazing, right? You could—if you’re co-founders, you could figure out how to divide equity. Unfortunately, it’s not that useful. It’s not that useful because in order to compute Shapley value, you need to simulate all worlds where some subset of the two of us collaborated but the third one doesn’t, and play out those realities to then impute back to today in terms of how we should divide the pie. Which in practice you can’t do in most places, right? In ML models, when you do feature importances, you kind of can. You can hold a feature back, run your model and see how well it did. In web search, we can run simulations of if we did not have access to this URL or this domain or this collection of them, how would the agent perform? We can—if you’re good at evals, if you’re good at assessing quality, you can build that data by running various scenarios, collect a bunch of this data, and then you can train models to ... Sonya Huang: Your favorite thing. Parag Agrawal: Favorite thing. So the challenge with Shapley values is computing Shapley values in our context is way more expensive than the amount of dollars we spend on the agent. Forget the amount of dollars we want to pay a publisher, right? So to compute a content owner, to compute that a content owner gets a dollar, if I decide to do the full Shapley value computation, that might take several dollars. So it doesn’t make any sense, except you can really well estimate them if you build the right kind of data and models around it. But we have confidence that our estimations are good and that this is sound theoretically, that if there was perfect information symmetry, people would want to collaborate. So the same way in the old ads days, people did second-price auctions and believed, for better or worse, that people would reveal their true bids in an ad auction and end up paying less than that, once feedback loops get established in a market—today, everything is like an auto bid, right? Most people are measuring ROI when they’re doing advertising and running on auto bid instead of making up bids. I think a solid foundation based on incentive alignment that Shapley math drives will ultimately maximize participation of content owners into this, as well as content seekers via AI in an optimized system. And we have some positive evidence to support it. We’ve had some interesting partnerships that we’ve been able to do and announce. And yeah, imagine sitting with content owners and explaining Shapley math—takes a moment. But ultimately, once you pull out the properties that you participate in the value, if you have unique differentiated data, you get paid more. If a banker in an expensive job reads your data versus my retired dad reads your data, the banker ends up paying more for that read, right? Because it’s a part of high-value work. And the macro math also seems to work. If you’re going to spend a lot of money on inference on LLMs for knowledge work, if we allocated what, two to ten percent of it to data on the web, that’s way bigger than all web data business models today outside of walled gardens like Facebook and LinkedIn. So the macro math supports it. It is a scalable approach, and as agents on the web grow an order of magnitude year on year, by my calculations, we’re 12 to 24 months from this math being able to give meaningful dollars for a very wide range of content owners on the web. Sonya Huang: That’s very exciting, especially at a time when you see the traffic data, the stack overflow plummeting. You see a lot of the human internet as we know it going away because of incentives. It’s very exciting to see how you’re thinking about incentive alignment for people to keep publishing. Parag Agrawal: Yeah, it’s why we started the company. Sonya Huang: Why is the company called Parallel? Parag Agrawal: [laughs] The company’s original name was Shapley Inc. Sonya Huang: Really? Parag Agrawal: It was. When ... Sonya Huang: Did you know this? Andrew Reed: I knew that, yeah. Sonya Huang: Huh! Parag Agrawal: When I incorporated, I’m telling you, I was obsessing about everything to do with the problem space. So while knowing that the first set of—also, one, Shapley Inc. is a terrible name for a B2B product. It was not going to be the long-term name. But I incorporated as Shapley Inc. for lack of a better word, and “shapley.ai” happened to be available. Shapley.com is a parked .com domain. So it was not going to be our name. I went around not talking about my company as Shapley, and my badges at all events used to call it “NewCo” or “Stealth Company.” And it took us almost six to eight months to figure out what the real name of the company will be. We ended up at Parallel in part because it’s one, at the time we were doing a lot more in parallel. And two, we started visualizing this sort of a parallel web built for AIs and how its properties are different. And this metaphor that when you publish, you’re now thinking of—as we all are now—of two audiences. Okay, I’m going to create a page. I know humans will read it. What should it look like to them? And then how should I make sure that agents can read it too? So it feels like you’re dual publishing to two audiences. And so we had this idea that a parallel web for agents will emerge. And so that’s why we started liking Parallel. Sonya Huang: Love it. Andrew Reed: I actually was thinking about this recently with things like earnings transcripts, right? Like, I feel very confident that more people are consuming earnings transcripts through agents than are actually certainly listening to the audio and honestly, probably reading the transcript itself. So there are these very high-value portions of the web that have flipped already to being agent first. And obviously, the communication has not yet flipped. But I think if I were a public company CEO today and I was doing an earnings report, I would make very clear that the thing that I’m saying will be transcribed and interpreted correctly by the agents, not just by the people listening, theoretically. Parag Agrawal: Yeah. Same for us when we’re publishing docs for our APIs. Our customers are building AI solutions. They are using AI to do it. It’s their agents reading our docs and code in our every case. It’s not humans fumbling around docs pages for the most part. So in fact for us, the primary audience is an agent, and that’s how we test our docs. Sonya Huang: A parallel web for agents. That’s very cool. Maybe close us out. Tell us—just give us a snapshot of where Parallel is today, and if everything that you hope to build comes true, what does Parallel look like? What does the world look like? What’s your role in it? Parag Agrawal: I think of three levels in this journey. Level one is people are building simple agents that use the web more as a tool, like a web search tool, because it’s familiar. Because the first set of agents we built, to your point, was just like, model and give it the same tools, because these models have been trained to use all the tools humans have been used to, and let them do work. By and large, if you think of most work, it’s there today, or inference, it’s there today. We have now started seeing some subset of customers who are in this world where they’re building more sophisticated multi-agent systems, which use subagents and have agents wake each other up or orchestrate in interesting ways, right? People are seeing that as subagents. The most familiar one is subagents within coding agent harnesses. But we see a lot more of that. Like, if you’re building an AI scientist, some of those systems are very interesting, very sophisticated, very long-running, and just throw large amounts of compute and data at a really hard problem. And I think the third layer for the web specifically is the web goes from pull to push. So today, by and large, across the first two modalities I described, either an agent is calling a tool or a sub-agent, but it is telling it a request saying, “Go find this for me right now.” And I think where we will end up in a couple of years is a variety of use cases will be the web, Parallel, call me if this happens, so that my agent can do some work or a human can do some work. So I use this line with my team all the time, which is like, imagine agents are everywhere and they can do a lot of things, right? If there is something you can do today and it’s worth doing, and we still have GPUs available, we’ll just go do it, right? We won’t be like, oh, let’s just do this tomorrow for the sake of it. If it can be done today, do it. So what will we do tomorrow? We will do tomorrow work in response to something that circles through either another agent’s work or something changes in the world as visible in satellite imagery, or some customer commentary that happens, some agent finishing some compute, a human having a new insight to trigger work. But there are going to be a few feeds like this which will drive new agentic work tomorrow. And one of those feeds is going to be everything that changed on the web, which is really exciting for me, because then you are framing not a point in time need, but a long-term here is what is actionable for me. If something like this happens as evidenced by all of the information on the web that is changing all the time, call me, right? And then I’ll run my agent on it. And so we get to then allocate compute onto the entire web all the time on behalf of all the customers. And that’s really exciting. Sonya Huang: That’s awesome. Really ambitious vision. You’re clearly extremely passionate about this and exciting to see you building. Thank you so much for joining us today, Parag. And thank you for joining us, Andrew. Andrew Reed: You’re very welcome, Sonya. Sonya Huang: Thanks, Parag. Thank you, Andrew, for your debut. <div class="framer-mcvrsd" id="