No. 95·中文精读

造东西越来越容易,知道造什么越来越难:一年百万场 AI 访谈的生意

Knowing What Your Customers Want, All the Time: Listen Labs' Alfred Wahlforss

Listen Labs 的 Alfred Wahlforss 谈用 AI 智能体同时做数千场语音访谈,替代焦点小组。

节目音频00:00 / 40:22
中文精读由 AI 基于原始访谈改写,非官方翻译,可能存在偏差或错误;表述以原节目为准。

一件衬衫卖不动,原因可能出在胸毛上。

短裤品牌 Chubbies 用 Listen Labs 做产品测试,AI 访谈挖出了这个细节:男用户的胸毛和某种面料凑在一起,穿起来难受得要命。衬衫改版之后,舒适度大幅提升。同一家公司还帮男士护理品牌 Manscaped 改过超级碗广告,帮 Skims 测过产品。

Listen Labs 是一家 AI 优先的客户调研公司,上线刚满一年,客户里已经有 20% 的财富 500 强,微软、Anthropic、Sweetgreen、NBC 都在名单上。它背后连着一个 3000 万人的受访池,AI 代理能同时跟几千人开视频访谈。到目前为止,平台上累计跑过 100 万场访谈,还在指数增长。

创始人 Alfred Wahlforss 最近上了红杉资本的播客 Training Data,两位主持人 Sonya Huang 和 Konstantine Buhler 都是红杉合伙人。Alfred 的核心判断浓缩成一句话:

「越接近 AGI,造东西会越容易,难的部分是知道该造什么。」

产品长什么样

流程不复杂。你输入一个问题,比如「怎么改进 Cursor 的新手引导」,系统先生成一份访谈指南,本质是给 AI 代理的指令。然后从 3000 万人的池子里找人,从肿瘤科医生到软件工程师都找得到,同时开几百场访谈,最后汇总分析,直接给建议。

访谈本身是一场视频通话,类似跟 AI 开个 Zoom。AI 会读你的表情,看你的眼神,听你说话的方式。整个平台围绕可溯源设计,每个数据点都能点回原始视频或原话,你能亲眼确认 AI 没有编造。

Alfred 反复强调一个事实:大多数公司的大多数决策,根本没有用户输入参与,因为找人聊用户的摩擦太大了。摩擦降下来,决策质量自然上去。速度优势很夸张,五分钟内就能拿到真人反馈,看着几百个人涌进你的访谈,体验相当魔幻。因为整个过程是异步的,成本也压下来了,付给受访者的钱比同步访谈少得多。

这个产品的起点挺偶然。Alfred 和联创之前做了个消费应用 BeFake,用微调过的 Stable Diffusion 给用户生成 AI 头像,算是 ChatGPT 图像功能的早期版本。应用一夜爆红,两万个用户涌进来,然后流失严重。他们给自己做了个 AI 访谈工具,想搞清楚用户为什么走。用下来发现这东西太好用了,干脆转型做 Listen。

对比一下旧世界。老牌问卷工具 Qualtrics 之外,市场调研还有一个几十亿美元规模的服务业,专门帮公司设计问题、招募受访者、分析几百小时的录音。传统消费品公司、微软这样的巨头,每年花几千万美元做焦点小组,把人请进房间聊。设计问卷本身是门学问,你不能直接问「你愿意为这个付多少钱」,不同问法对行为的预测力差得很远。

问卷早就不可靠了,然后呢

主持人 Sonya Huang 的质疑很代表性:受访者是拿钱办事的,天生有选择偏差;人说自己会怎么做,和实际怎么做,经常是两回事。她更相信真实世界的行为数据。

Alfred 的回应分三层。

第一层,一致性测试。他们把同一道选择题问卷隔段时间再发给同一个人,答案差异很大。换成 Listen 的语音访谈,人得真正动脑推理,前后一致性明显更高。

第二层,回测。Chubbies 用他们测所有东西,从营销素材到衬衫款式,测完之后几个月,拿真实销售数据回来对答案,看访谈预测准不准。

第三层,他承认 A/B 测试是黄金标准,但实践中很难做对,需要足够大的用户量。有一点真实输入,总比没有任何输入强。

表情识别是弥合「嘴上说」和「实际想」之间差距的另一条路。量表上给你打高分的人很多,但真正眼睛发亮、语气兴奋的反应,才预示这支广告在 Meta 和 LinkedIn 的实际投放里表现更好。他们验证过这层相关性。

人更愿意对 AI 说真话,找对人也更难了

人们居然愿意接受更低的报酬来跟 AI 聊天,换成真人访谈员就得加钱。原因是异步、随时开始随时停、心理压力小。有人聊着聊着把心里话全倒出来,像做了一次心理疏导,对面是个不带评判、又对你充满好奇的实体。他们甚至能做儿童调研,这在过去是出了名的难,要家长同意,孩子要上学、上兴趣班,时间根本约不上。

另一件花大力气的事是找人。Listen 把 80% 的工程资源砸在受众这一端。客户分群是幂律分布的。Sweetgreen 看起来是给所有人做的沙拉,真正的核心客群是城市高收入女性,还得知道「种子油」是什么,那是欧美健康圈的热词,全美国只有大约 1% 的人知道这个概念。每天去吃的那批人,贡献了 80% 的收入。找到这群人,调研结论才有可执行性。

Alfred 的目标是把受访池做到 10 亿人,给每个人标注专长。可能是个球鞋发烧友,品牌发新品前先访谈这批早期采用者,洞察价值高得多。每做一场访谈,这个人的画像就更厚一层,下次直接搜出来。Konstantine 补了一句,这背后有网络效应,规模到了之后,那些平时极难接触到的人,你也能约到了。

旧模式做不到这一点。传统调研数据库有个指标叫发生率,可能只有 10%,意思是十个人里只有一个合格。被筛掉十次才拿到第一次报酬,数据库流失严重。直接用自家 CRM 也不省心,数据乱是一回事,谷歌不能随便给 Gmail 用户发调研邮件,普通公司群发问卷容易被当成垃圾邮件,直接进过滤器。接 CRM 发给熟客当然可以,但更有价值的是触达那些还不是重度用户的潜在客户,再把两拨人的答案放在一起对比。

调研 3.0:模拟你的用户

Alfred 把行业分成三个阶段。1.0 是人工打一百个电话,手工整理。2.0 是 AI 设计问题、同时聊几千人、自动汇总,Listen 现在干的事。3.0 是生成式代理模拟,几个月内上线。

模拟的原理:给一个人做一小时深度访谈,就能开始预测他的偏好,LLM 很擅长这个。某些场景下,预测这个人怎么回答问题的准确率能到 95%。

麻烦在于世界一直在变。混沌理论告诉我们,预测未来很难,否则大家都去华尔街赚钱了。所以要用持续流动的真实访谈给受众「补水」,平台上累计的 100 万场访谈就是燃料。Alfred 描绘的场景是:你问「软件工程师怎么看 Claude Code」,Listen 回答,我这周已经访谈过 1000 个软件工程师了,我来预测他们会怎么答。

怎么知道模型靠不靠谱?他们做回测:从训练集里抽掉一道题,看预测得准不准;再塞进「他们的狗叫什么名字」这种无厘头问题,看模型知不知道自己预测不了。

Alfred 自己先当了小白鼠。他要给一场演讲起标题,写了 100 个,丢进模拟面板,第一名比第二名好一倍。他又做了个对照,把两组演讲标题分别给 ChatGPT 和自家模拟挑,ChatGPT 挑错了,模拟挑对了。

原因在哪?通用模型学的是「平均人」,而你要服务的是一个非常具体的窄人群。他们也试过信用卡消费、行为数据,效果最好的数据源还是访谈,因为访谈能跑题、能追问行为、能捕捉一个人怎么想。模型缺的是关于具体人群如何行动的干净数据。

访谈攒够之后,产品形态也变了。他们有个功能叫增强响应(augmented responses),相当于给你一个专属 bot,随时问即时问题。技术栈是后训练加 RAG,再加一些不公开的私招。它还能接进你的编码代理,Alfred 称之为 human API,代理随时调用用户偏好,知道该造什么。他们已经发布了 MCP,接进 Claude 之后,可以让 Claude 在循环里跑 Listen,批量生成营销创意再测试。

Sonya 追问,要不要让一千个模拟用户互相辩论,类似把同一个问题发给三个模型再让一个模型当裁判。Alfred 持怀疑态度,事物一复合就难预测,混沌理论又来了。他们的做法是把一个人建模建好,再扩展成一千人的代表性样本。

生意层面的数字更有意思。因为做得快,反而应该收更多钱。他们向客户收几十万美元,对应的只是一场覆盖八个国家、20 位医生的访谈项目。长期看,单场访谈会更便宜,但研究总量会多两个数量级。模拟解锁的是那 99% 你永远没时间找真人的场景。分工很清晰,超级碗广告这种大决策还是要跑真人访谈,广告牌上的标语这种小决策交给模拟。咨询公司也在用,Bain 拿 Listen 加速传统流程,但利润率会掉,得把服务拆开,让 AI 代理接走一部分。

Konstantine 给了一个医疗场景:医疗系统里有一堆小毛病,医生忙到没空参加任何座谈,公司也懒得花时间搞清楚那些细碎的痛点。如果医生在手机上顺手就能吐槽 EHR 或者手术室里的设备,这类场景听着就能救命。

护城河是笨功夫

创业初期的核心想法很朴素:把访谈体验变得不那么烦人。GPT-4 时代,AI 有时会把同一个问题问一百遍。他们建了一个评测,专查重复提问和是否遵循指令,起步 20%,一路爬到 85%。然后建了一个更难的评测,要求 AI 理解屏幕录制里你在干什么、能不能跳过不再相关的问题,又掉回 20%。

垂直 AI 公司的优势就在于拥有自己的私有评测,然后一点点爬它。这是 Alfred 总结的方法论。

护城河有几层:受访池的网络效应,供需两端互相吸引;数据护城河,访谈越多模拟越准;产品粘性,历史访谈都存在平台上,你要长期追踪。还有最朴素的,红杉合伙人 Bryan Schreier 跟他说过一句话,创始人想做复杂的东西,客户只想要傻简单、能用就行。

生成访谈指南就是典型。早期他们直接用裸 LLM,客户拿到数据跑回来说这玩意没法用,他们把这个锅背下来,训练模型遵循最佳实践,比如别诱导受访者。

这门手艺的历史战绩不小。P&G 通过客户访谈发现洗衣液用着难受,用户想要更省事的形态,做出了 Tide Pods 洗衣凝珠。Mars 在上世纪 50 年代做了最早的调研之一,M&M's 原本给军队设计,因为揣兜里不化,调研发现孩子是更大的市场,整个广告策略随之转向,这个故事出自 Acquired 播客。大公司的账更好算,宝洁进入一个新市场,投入动辄几千万美元起步,发布那一刻必须对,这是他们成为 Listen 大客户的原因之一。

Konstantine 还给了一个视角:公司的本质是服务人,AI 越聪明,AI 不知道而人知道的那部分差值越值钱。Alfred 确信未来仍需要真人输入,因为人是不理性的,会突然迷上一个新的 TikTok 趋势,逼你整个营销策略掉头。他不确定的是模拟能走多远。

Sonya 把商业模式总结成一个无限循环:公司本质上就是「搞清楚造什么,造出来」来回转,AI 正在把「造出来」这半边推得飞快,Listen 卡位的是「搞清楚造什么」,从产品功能到广告文案。Alfred 补了一个更朴素的版本:写代码,跟用户聊。终局画面是一个人的十亿美元公司,编码代理负责写代码,Listen 负责搞清楚该写什么,两个在循环里跑,自主运转。

「我们希望活在一个终于按人们想要的方式运转的世界里。」

明天就能用上的几件事

  • 设计问卷时,删掉「你愿意付多少钱」这类直接问法,换成行为类问题,问过去做过什么,比问将来会做什么靠谱。
  • 标题、广告语这类小决策,批量生成 100 个版本,找一批匹配目标画像的受访者测试,选得分最高的。Alfred 的演讲标题就是这么选的。
  • 看任何调研报告,先要求每个结论能点回原始录像或原话,不能溯源的数据先当不存在。
  • 如果你在做 AI 产品,建一个自己的评测集,专挑你最怕出的问题,每次换模型跑一遍,分数就是产品进度条。
  • 预算有限时,优先调研核心客群。那群看起来很小的人,可能贡献 80% 的收入。

最后回到那句判断。造东西的成本正在塌方,知道造什么的成本正在变成新的瓶颈。谁先建立起持续听用户说话的管道,谁就握住了循环里更难的那一半。

文稿来源:sequoiacap.com;机器转写可能有识别错误,人名与数字请以音频为准。
id="podcast-transcript" data-framer-component-type="RichTextContainer" style="transform:none">IntroAlfred Wahlforss: Our goal is to get to a billion people in our audience and then to be able to stratify and know what exactly is this person an expert on. And it might be, you know, even something like sneakers. You have some people who are influencers and kind of early adopters. And if you’re able to find that audience and interview them first, the insights are much more valuable. And we can learn across all of the interviews that we do. We build profiles of people as we do more interviews in the platform, and then we can search and find the right person. Main conversationSonya Huang: Okay, today we’re sitting down with Alfred Wahlforss, founder and CEO of Listen Labs. Listen is an AI-first customer research platform that can run thousands of voice interviews simultaneously. You launched about a year ago, and you now serve 20 percent of the Fortune 500, including iconic brands like Microsoft, Anthropic, Sweetgreen, NBC and others. And Konstantine and I are very, very excited to sit down with you today and talk about market research and how it’s getting transformed with AI. Alfred Wahlforss: Yeah, thank you for having me. Sonya Huang: Maybe just to get started—you are building an AI-enabled platform that scales market research. What does that mean? Alfred Wahlforss: Yeah, so we have this AI agent that can understand your customers better than you can. And the way we do that is by talking to them. So to give you an example, you can ask a question like, “How can you improve Cursor’s onboarding?” And then Listen will create an interview guide, which is instructions for the agent to conduct the interviews. And then we have an audience—we have 30 million participants. We can find pretty much anyone from an oncologist to a software engineer, and we’ll go and actually talk to them and have hundreds of those interviews, and then analyze the data and give you recommendations. And now the final step that we’re just launching in a couple of months is simulation. So after you’ve done tens of thousands of interviews in the platform, can you predict how your customers will answer questions in the future? To put it another way, as we get closer to AGI, it will be easier to build things, but the hard part will be knowing what to build. And that’s what we’re building at Listen. Sonya Huang: Awesome. Do you have any favorite customer stories? Alfred Wahlforss: Yeah, so Chubbies is one of our customers. Sonya Huang: Like the shorts brand? Alfred Wahlforss: Yeah, they’ve been one of our early customers. Sonya Huang: What do they use you for? Alfred Wahlforss: They use us for everything—so a lot of marketing testing, testing shirts to understand what products perform well and what doesn’t. And one of my favorite examples is they discovered that chest hair interfaced really poorly with one of the materials they have, so it’s really uncomfortable to wear one of their shirts. And they changed the shirt and it became radically more comfortable. So we solve the small things to the big things. Manscaped changed their Super Bowl ad with insights from Listen. Sonya Huang: Never heard of that, but I’m not going to ask. Konstantine Buhler: So you’ve got the men’s hair market covered. Alfred Wahlforss: [laughs] Yes, that’s our niche. Konstantine Buhler: From shaping to clothing. Alfred Wahlforss: That’s right. Konstantine Buhler: Wow. Alfred Wahlforss: We do other things. Skims is one of our customers as well. [CROSSTALK] Sonya Huang: Don’t know what you’re talking about, but I know context clues. So that’s awesome. I’d love to understand—as you framed it, as we get closer to this AGI future, one of the questions I have is: Traditionally I’ve always been very skeptical actually of surveys, because people get paid to take surveys, so you already have a selection bias issue. The things that people say they would do or the way that they describe how they would behave is different from how they actually behave in practice. And so I come from the school of thought where actual telemetry in the real world matters so much more than asking people about what they would do. And so I’m curious what you think of that, and how you think AI or Listen Labs can help bridge that gap. Alfred Wahlforss: Yeah, so we’ve done a lot of research on this. One of the things we’ve done with surveys, for example, is we went back to the same person and asked them a multiple choice survey again, and they were radically inconsistent. So even if you go back to the same person and ask them a survey question in a multiple choice fashion, they’re much more inconsistent. But we did the same thing with Listen, when you actually have to think and really reason through your answer, you’re much more consistent with at least how you answer the same question. And then we are constantly tracking—for example with Chubbies, when we test their different shirts, we a couple of months later look back and see how did that perform with the actual sales data. And I think it depends on the different use cases. I agree that A/B testing is kind of the holy grail, but in practice it becomes really difficult to get right because you need a very large volume of users. And it’s really useful to have some kind of input rather than no input at all. Konstantine Buhler: Does Listen do voice to text, as in the actual customer who’s answering the survey can speak their answer and then you guys transcribe it? Does it also do text to voice? Is it a two-way conversation? What does Listen start with and what does it finish with for the user experience? Alfred Wahlforss: Yeah, so it’s essentially a Zoom call that you have with the agent, so you’re on video. And we can also detect your emotions. Konstantine Buhler: Hmm, that’s cool. Alfred Wahlforss: So that’s another way to bridge the gap between what they say and how they actually think and feel. So it looks at your eyes, the way you say it. and that’s kind of much closer to how you actually behave in the real world. Konstantine Buhler: And have you seen, per Sonya’s point, that actually having the person’s face and their emotions and their voice and whatnot yields more engagement and truthfulness? Have we been able to have any studies or at least data to point in that direction? Alfred Wahlforss: Yeah, specifically with advertising it’s a huge benefit, because you might have people, say, on a Likert scale, which is like five questions that you click are you extremely likely to buy this product, versus you might have very high scores on a certain question like that. But when someone also reacts very enthusiastically, it’s going to perform much higher. And we’ve seen that those ads then perform better in performance marketing for example on Meta and LinkedIn. Konstantine Buhler: And can you, if you’re the customer and you commissioned this and you get all this response, can you actually  click in and if you ever wanted to, watch the interview to get that level of granularity? Alfred Wahlforss: Yeah, so we built the platform around traceability so that for every data point you can always click and then look at the video or see the quote, and so you know that the AI is not just hallucinating—you can see where it’s coming from. Konstantine Buhler: That’s awesome. Makes sense. Sonya Huang: How’d you come up with the idea to build this? Alfred Wahlforss: So my co-founder and I actually built a consumer app. Sonya Huang: That did what? Alfred Wahlforss: That went viral. It was called BeFake. So you could create an AI avatar of yourself. It was an early version of the ChatGPT images, and you could fine-tune Stable Diffusion and put yourself in that world. And that ended up going super viral, and overnight we had 20,000 users. And we were also experimenting with different ways of using AI, so we built this AI interview for ourselves because we had a bunch of questions of how—we had a ton of churn, so we wanted to understand why, how they thought about our positioning, different use cases. And it was really useful for us, and that’s how we got started. Sonya Huang: Maybe just walk us through how the industry is changing before and after Listen Labs. Historically, let’s say you’re somebody with an app with 20,000 users. You don’t understand how users are using the app, what they want next, why they’re churning. Historically, how did people go about doing that? Alfred Wahlforss: Yeah, so what we discovered was that there are these survey tools that are pretty old school, like Qualtrics. But then there’s also this very large services industry, because it becomes harder and harder—especially if you want to do market research where you want to talk to your prospective customers, not your current customers, it becomes harder and harder to do that as you scale. So that’s a multi-billion-dollar industry. And what they do is come up with questions to ask, which is an academic subject in and of itself. It’s actually really hard to know, like, how do you ask questions to your point that get to how someone actually will behave? You can’t just ask, like, how much are you willing to pay for this? There are different methodologies that work better than others to finding the audience, how do you source the participants, to then analyzing hundreds of those calls. And in traditional industries like CPG, even at Microsoft, they spend tens of millions of dollars on focus groups, to bring people into a room and interview them. We can help speed that up much faster. Sonya Huang: Okay, so that’s the old world of how this used to be done. Maybe describe the new world. And then it seems to me that there were obvious first-order benefits, like it’s probably much more scalable, probably much more cost-effective. But there are probably also less obvious benefits. Maybe just talk about some of the benefits of, you know, what is it like when you actually do AI-first market or customer research? Alfred Wahlforss: Yeah, so most decisions that get made are not based on customer input, right? And the reason for that is it’s just a lot of friction to even talk to customers. So when you can lower the barriers of talking to customers, you end up making much smarter decisions. So the speed advantage is actually huge. For us, you can get input within five minutes from real people. And it’s a really magical experience when you see hundreds of people populate in your interview. And so that’s one thing. And because it’s asynchronous, it’s also much more affordable, so you can pay people much less than if you would’ve had to run synchronous interviews. And so actually that’s an interesting thing that people often ask us, like, do people even like being interviewed by an AI? And the objective answer is yes, because you can pay them less to talk to an AI than to talk to an actual interviewer. Sonya Huang: Why is that? Alfred Wahlforss: I think it’s mostly because it’s asynchronous and people are very busy, but then also … Konstantine Buhler: Lower pressure? Alfred Wahlforss: Yeah, lower pressure. You can kind of go on and off. And we’ve also found that people are more honest when talking to an AI. We’ve had people really open up. It’s a very therapeutic experience, because it’s a non-judgmental entity that’s really interested in you. And we can also have sensitive conversations, like interviewing kids about how they react to different products. And so I think that’s another advantage as well is that people can be brutally honest talking to the AI. Sonya Huang: Okay, so historically, for example, if you want to do research on the kids’ market—very, very hard to access that market. Is that a regulatory thing? Is it a scheduling thing? Alfred Wahlforss: Yeah, it’s—you need parental consent. Kids are really busy: they go to school, they have extracurricular activities. How do you find time with them? And you need to find the right kind of kids. Like, one of the things we realized is the audience is extremely important, and that’s actually where we spend 80 percent of our engineering resources. Every company is driven by a power law in customer segmentation. So even a product like Sweetgreen, which you would think is for everyone, the right audience is typically urban, high household income, mostly female. And by the way, they need to know what seed oils are, which only about one percent of the population does. And then you find that some people go to Sweetgreen every single day and that’s 80 percent of their revenue. So if you can find that segment, the research is so much more actionable. Konstantine Buhler: Yeah, there’s probably a network effect to it as well, where once you get to a certain scale and people use it, you can access the same kind of person that otherwise might be really difficult to access. Or maybe it’s a scale economy—something along the lines of accessing those really, really specific people who are really valuable for the type of product you’re trying to introduce. Alfred Wahlforss: Yeah, it’s really about—our goal is to get to a billion people in our audience, and then to be able to stratify and know what exactly is this person an expert on. And it might be even something like sneakers. You have some people who are influencers and kind of early adopters, and if you’re able to find that audience and interview them first, the insights are much more valuable. And we can learn across all of the interviews that we do. So we build profiles of people as we do more interviews in the platform, and then we can search and find the right person. Konstantine Buhler: So someone might say in a totally unrelated interview, “I’m a total sneakerhead.” And you can keep that in the database on that person. And then when Nike or what have you is launching a new product line, you can offer that person up. Alfred Wahlforss: That’s right. Konstantine Buhler: That’s amazing. Alfred Wahlforss: And that was not possible to do before, because it was usually separate entities, and it would be a very manual process where you would have an email list and you would just spam email. Konstantine Buhler: I’ve been on the receiving end of those. Yeah, they’re terrible. Alfred Wahlforss: And one of the problems with that is you then need to have an extensive screening process. So you have something called an “incidence rate,” which can be 10 percent, which means only 1 in 10 people gets qualified to even take the interview. And that causes significant churn on these databases. Konstantine Buhler: Yeah. Alfred Wahlforss: Because it’s really annoying to be screened out 10 times to even get paid the first time. Sonya Huang: Why do these brands even need you for access? If we take Sweetgreen—Sweetgreen knows who that 80 percent is. Can’t they just reach them? Don’t they have direct relationships with them already? Alfred Wahlforss: Yeah, so they can—and we do that as well, we connect to their CRM and they can send that out. But then the really interesting part is how do you talk to prospective customers—people who also may not be kind of current power law users? And how do you compare those two? And then also what we’ve found is that the CRM is typically really unorganized. And sometimes there’s also regulatory issues—if you’re at Google, you can’t just send emails to people who use Gmail. And it gets much easier to use an external third party. Konstantine Buhler: And you run the risk of spam, which can get you totally blocked. I have seen that at some of our companies over the years where you do outbound, and then eventually you’re in the Google filter and next thing you know you’re in Microsoft purgatory. I guess going through you guys, you don’t have to deal with that. Alfred Wahlforss: Yeah, exactly. Konstantine Buhler: That’s cool. Sonya Huang: What does this mean for the McKinseys or the whoevers of the world—the people that are building the hundred-slide decks that, you know, reach 3,000 people to reach some set of incentives? Konstantine Buhler: Didn’t you do that, Sonya? Wasn’t that a former life? Sonya Huang: You know, no, Konstantine, but I’m glad that’s what you think of me. Alfred Wahlforss: [laughs] Konstantine Buhler: Isn’t that what [inaudible] does? Sonya Huang: We hired a consultant. I used to hire these people. Konstantine Buhler: Got it. Sonya Huang: So I was a layer on top of the layer on top of the layer. Konstantine Buhler: Oh, okay. Sonya Huang: I was even more redundant. But what does it mean for all these people? Do they still have a role to play in this new future? Alfred Wahlforss: Yeah, I think AI is changing all roles very quickly. And we work a lot with Bain, for example. So they use us to speed up their traditional processes. And I think they still have a role to play. I think traditional services and then being able to implement these changes is still extremely valuable. But a lot of margins are going to drop, and you have to make sure you kind of unbundle a lot of your services to maybe allow for AI agents to help solve some of the problems that you would go to traditional consulting firms for. Konstantine Buhler: Maybe I’m an optimist here, but why wouldn’t it be more? Why wouldn’t I, if I’m running a business, say, “Oh great, I want to find five new areas to expand into now that I have AI and these tools, and I will pay you, Bain or what have you, the same dollars. You use Listen, and just explore those new areas and tell me where to build.” Is that overly optimistic? Alfred Wahlforss: No, I think it’s one of those areas where the ceiling is very high. You can learn more about your customers and you can build more things. So I think you’re right. Konstantine Buhler: Yeah. I’m still thinking about the chest hair-shirt thing. Alfred Wahlforss: [laughs] Sonya Huang: I’m glad that 20 minutes in, you’re still thinking about chest hair, Konstantine. Konstantine Buhler: There are so many little things I’d love to tell the companies that I’m a consumer of—like, even the way they laced these shoes, I’d love to give that feedback. Sonya Huang: This is why you’re a venture capitalist. Konstantine Buhler: Details. Details. Alfred Wahlforss: We hope to live in a world that finally works the way people want. Konstantine Buhler: That would be great. Please. Sonya Huang: Are you seeing any pricing compression already hit the industry? Like, I would imagine if I am Bain’s customer, I’m thinking, well, you’re able to do this survey a lot more efficiently now with AI than before AI. Who’s getting the benefit of that economic surplus? Alfred Wahlforss: So because you’re able to do it faster, I would argue you should be able to charge more for it. Sonya Huang: And is that what’s actually playing out? Alfred Wahlforss: We’ve done some studies where we’re able to charge hundreds of thousands of dollars to speak to twenty doctors across eight countries. So maybe over the long term, like, the individual interview will become more affordable, but I think you’ll be doing two orders of magnitude more research. And I think what’s really exciting is also simulation, which is something we’re building now, where you’re able to unlock the 99 percent of use cases where you would never have time to talk to real people. Konstantine Buhler: I think that’s so awesome in part because there are so many areas where they don’t even listen to the customer. Like medicine. There are a million little problems with the medical system. I hear about it all the time. And these are doctors, you know, they’re busy, important people, but it feels like the companies haven’t even invested the time in figuring out where all those paper cuts are. And the doctors are really busy, so they’re not going to go schedule an appointment and have some long conversation and meet with some group. But if they could do it at any time in an app on their phone as part of their normal homepage app, and give feedback on their EHR or something in the operating room or something along those lines, that seems like a life-saving use case for Listen over time. Alfred Wahlforss: Yeah, I think what I’m really excited about as well is taking all those small things and then telling another agent to go and solve that problem. And we’re getting pulled in this direction by some of our customers where they will have a churn interview and then they will connect—if you find a bug for example, they’ll connect that to another coding agent to go and solve the problem. Konstantine Buhler: That’s cool. Sonya Huang: Let’s talk about generative agent simulation. It seems like the entire industry has gone from Market Research 1.0—call 100 people one by one, collate them manually—to Market Research 2.0, AI-native, where AI designs the question track, is able to talk to thousands of people simultaneously, synthesize the answers. It seems like we’re maybe moving to Market Research 3.0 with generative agent simulation. What do you make of that? You know, I both see the dream of it—I see how synthetic data has changed, for example, self-driving cars. And then I’m also inherently skeptical of it. Like, is a bunch of metadata just remixing what’s already in the pre-training sets? Are you actually learning anything useful or what else is in there? So I’d love to hear your take on it and how you guys are taking on the 3.0. Alfred Wahlforss: Yeah. And maybe what is it too, to start? Alfred Wahlforss: Yeah, so the way we are building simulation is by interviewing a single person. Say if I interview Konstantine for one hour, I can probably start to predict your preferences to some degree. Konstantine Buhler: Fascinating insights about chest hair. Alfred Wahlforss: [laughs] And it turns out that LLMs are quite good at this as well. So you can essentially try to feed in as much information as possible on a single individual. And then in some cases we’re able to get 95 percent accuracy to predict how they will answer certain questions. Now the problem becomes things are changing all the time, and chaos theory tells us it’s really hard to predict the future, otherwise we would be on Wall Street and making a ton of money. So the way we think about it is you need to hydrate these audiences. And the way we do that is through all of the interviews that are running through Listen. So we have a very strong network effect. We’ve done a million interviews so far, and that has grown exponentially since we reported that number. Konstantine Buhler: Wow. Alfred Wahlforss: And we’re able to train audiences on those interviews. So you can imagine a future where you can ask a question in Listen like, “How do software engineers think about Claude Code?” And then Listen will say, “Well, I already talked to 1,000 software engineers this week, let me predict how they’re going to answer that question.” But the tricky part is knowing what things you can answer and what can’t you answer, because … Sonya Huang: And how do you do that? Alfred Wahlforss: Yeah, we try to be very explicit to the model about what the domain of knowledge they have and then see how much you can expand that domain. That’s kind of the fundamental idea. And we can back test how well the simulation works with what’s in our training dataset, so we remove one of the questions and then see how okay, how accurately did you predict that. And then you can add in nonsensical things, like what’s the name of their dog or something like that, and then can say is the model able to understand that you can’t predict that? Konstantine Buhler: That’s really cool. Sonya Huang: What sorts of things are you finding you can predict well versus can’t? Alfred Wahlforss: One of the most useful things is message testing. So that’s the idea of, like, what’s the tagline on the billboard? I was actually using it this weekend. So I created a panel of our customer base, and I had to come up with the title for a talk at a conference. And it’s like a small thing, but it actually does matter because it will increase conversion if people show up. And I came up with 100 different titles for my talk and inputted that into our simulation. Konstantine Buhler: Oh wow. Alfred Wahlforss: And the top talk was like twice as good as the next one. Konstantine Buhler: Wow, cool. Alfred Wahlforss: And I don’t know if it’s correct, but it certainly felt correct, and it was really helpful to have guidance in making that decision. And I also think even if it’s wrong, it’s just nice to have some help in making a decision. It’s also nice to outsource your decisions. Sonya Huang: And how does it compare to just asking ChatGPT the same thing? Alfred Wahlforss: Yeah, so then I inputted the same questions into ChatGPT. I had another talk I did that was not so successful, and I inputted a competitor’s—or another talk that was more successful. And I showed both of them to ChatGPT, and both of them to our simulation. And ChatGPT picked the wrong one, and our simulation picked the right one. So it’s early for us—we’re going to release this in a couple of months—but it seems like it’s performing better than the general models. And the models are trained on the average person, and you want to build for a very specific niche, and that’s how we can essentially train the models to follow that niche. Sonya Huang: And just to push on this—because I think it’s so fascinating—can’t you kind of force the models into a specific niche or personality? Like, “Hey ChatGPT, you’re a 35-year-old, really grumpy software engineer that likes using your terminal.” And then it does take on the preferences of that niche. That’s sort of my mental model, at least. And so I’m actually surprised that ChatGPT wasn’t able to arrive at the right answer and that bootstrapping off real user data was, because ultimately it all is kind of a reflection of real user data, right? And so actually, what is the intuition for why sim-only on pre-trained data isn’t sufficient? Alfred Wahlforss: Yeah, so we’ve tried many different inputs, and that certainly performs a little bit better than just vanilla ChatGPT. But what performs much better is—we tried credit card spend, behavioral data, purchasing behavior, but what we found was the best dataset is interviews, because it allows you to go off tangents, it understands—you can ask behavioral questions. And also it can’t just be any interview—like, the way you design the questions is also really important. And the intuition, I think, is that the models don’t have clean data on how a specific persona acts and how they think. Konstantine Buhler: It’s anecdotal, but it makes perfect sense, because if you want to understand someone, what better way to understand them than asking them a lot of questions? That’s why we’re all here—it’s kind of the purpose of this kind of format. And if you have enough people that follow a certain group as opposed to the average, that can tell you a lot about other things they might not have explicitly said. All of AI is this generalization of some sort of compressed data, of some sort, and so if you have this compression in a slightly different part of the hyperspace, and you say, “Now complete this orbital of what everybody is thinking in this category of person,” Listen can fill that out because it has enough interviews. Alfred Wahlforss: Yeah. Konstantine Buhler: Do you think you’ll offer that package as a product? As in if I wanted to understand my customer—and for me, for us our customers are founders, and they’re very different people. If I wanted to understand my customer, could you do active interviews, the normal Listen Labs interviews, have 1,000 or 10,000 cumulatively, and then offer a little special-purpose Listen Labs bot that then I can use instantaneously for any ad hoc question? Alfred Wahlforss: Yeah, that’s exactly what we have. So that’s what we call “augmented responses.” The cool part of this as well is that it can also live in your coding agents or your other agents. So I think in the future you’ll want to have almost a human API where the agents are able to call the preferences of your users to be able to know what to build, how to do it, or who to invest in, or how to help them best. Konstantine Buhler: Today, is it all RAG? Is it fine-tuned? Is it something else? How do you take those conversations and then combine them with the models you’re doing the rest of Listen Labs with? Alfred Wahlforss: Yeah, we are doing post-training, typical RAG as well. There are a bunch of different techniques. Some of them are proprietary, but yeah. Konstantine Buhler: All right. We’ll do customer interviews on all your engineers. Report back. Sonya Huang: I’m curious what you think of multi-agent systems and their role in helping us kind of iteratively use—at inference time iterate a better answer. Is that part of how you’re doing simulation or not? Alfred Wahlforss: Yeah, the way we do simulation is essentially you have one person that you model really, really well and then you scale it up with a thousand people. So you have a representative sample, and it’s essentially multi-agent. Sonya Huang: But you’re not having those thousand people debate each other. That’s what I’m asking. Alfred Wahlforss: Oh yeah. No, we don’t have that yet, but that’s a good question. Sonya Huang: Do you think that would help? Alfred Wahlforss: Potentially, but there are these other competitors that are doing more of that approach. The worry is that again, chaos theory tells us that when things compound it becomes really hard to predict how they’re going to interact with each other. It’s something we should definitely explore more, but I’m a little bit skeptical of the approach. Sonya Huang: Maybe the analogy I’d make is the AI council approach, which is send the same query out to three different LLMs and then have one LLM act as judge in synthesizing them. I do think on average you get a slightly better response. Alfred Wahlforss: Yeah. Sonya Huang: Cool. So where else do you see yourself going from here then? You’re going from Market Research 2.0 to Market Research 3.0, now with kind of generative simulation. Do you expect that 3.0 takes over as the majority of queries over time? And then what else is ahead? Alfred Wahlforss: Yeah, I think you’ll still need human input. But I think there would be many more use cases that are now opened up where you can get customer input. So for the large decisions—if you’re doing a Super Bowl ad or things like that—you’ll still need to run real interviews. But for the smaller things, like what should be the tagline for your billboard and it’s a small billboard, then you can use simulation to answer that. And I still think there’s a lot of alpha on the core product as well to improve. I mean, when we started, the core idea was just making the interview less annoying to go through. We had an eval that looked at repetitive questions, or looked at is the AI even able to follow the instructions. And with GPT-4, sometimes we would ask the same question a hundred times. Sonya Huang: Yeah. Alfred Wahlforss: And in the beginning, that eval was at 20 percent. Now we’ve been able to climb that eval to be 85 percent. But now we created a new eval that’s much more advanced, so it’s able to understand what are you doing on your screen when you’re screen recording, or can you skip questions that are not relevant anymore. And now we’re back at 20 percent, which I think that’s one of the values that vertical AI companies can have is that they have this proprietary eval that they can use and essentially climb that eval. And that’s your advantage as a vertical AI company. Konstantine Buhler: Keep pushing forward—better data, harder problems, better data, repeat. Alfred Wahlforss: Yeah. Sonya Huang: It seems to me like you’re in the middle of a very interesting infinity loop, right? Because fundamentally a company is: figure out what to build, build it, figure out what to build, build it. Alfred Wahlforss: Write code and talk to users. Sonya Huang: Exactly. And the “build it” is coming up rapidly in exponential. And the “figure out what to build” is the thing that you are pushing forward. And then not only—even outside of product and engineering, the broader loop is actually strategy, execution, strategy, execution. And so much of what AI is enabling us to do is making execution faster, cheaper, better, all these things. And the thing that you guys are fundamentally positioning yourselves to do at the company is the strategy part—from what to build to what to say. Is that a fair synthesis? Alfred Wahlforss: Yeah. And I think when we have that one-person billion-dollar company, we’ll be part of that loop. So you have a coding agent and Listen and then run that in a loop. And we’ll have these autonomous organizations. And you can do that. Konstantine Buhler: Even the big companies, though. Like, back to this idea of you can implement things faster. Let’s say you have an agent—I mean, if you can be a big company—and we’re talking in software because software is native to us—in software, if you could talk to a customer, figure out a bug, create a PR, have a coding agent close it, ship it, customer’s happy, that seems like a really important left-hand side of the equation—find the bug from an actual human. But I imagine it’s the same thing in a big atoms company. Like, if you’re consumer packaged goods, if you’re clothing, if you’re any of those things, I imagine it’s even more important because once you actually do the thing, it’s done. Alfred Wahlforss: Yeah, exactly. Procter & Gamble, when they’re launching in a new market, that can be tens of millions of dollars, if not more. And you have to make sure that it’s right when you launch. And that’s one of the reasons why they’re big customers of Listen. Konstantine Buhler: Who has done this historically really well? Who are the companies that are admired in history that have done a great job of listening to their customers, either in the consumer space or in the software space? Alfred Wahlforss: I think Procter & Gamble is kind of the archetype of the best market research organization, where there are essentially marketing companies that are trying to figure out what are niches that people really care about, and then build specific brands to solve those problems. I mean, one example is the Tide Pods. They were able to figure out that it was really uncomfortable to use the washing liquid, and discovered that people wanted something that was much more easier to use. And through customer interviews they found that insight, made the Tide Pod, and it became really successful. Another example, which is in the Acquired podcast, when they talk about Mars, they did one of the first market research studies in the 1950s where M&M’s were originally designed for the army because they were a sweet treat that doesn’t melt in your pocket. And they discovered through market research that another great segment was young kids. And they then decided to pivot the entire advertising strategy to focus on this, because it doesn’t melt and ruin your furniture and things like that. Sonya Huang: As we progress towards this Listen Labs future vision of the world, what are the things that you’re confident will work and what are things that you’re still not sure about? Alfred Wahlforss: I’m confident that in the future you’ll still need to have human input, because even if you have a perfect rational being like AGI, humans are still irrational. Konstantine Buhler: Totally. Alfred Wahlforss: And they will still be chaotic in their nature where they all of a sudden getting obsessed with a new product, a new TikTok trend that shows up and you have to change your entire marketing strategy towards that. And so I think that will remain a really huge part of how we do things. I think I’m still uncertain about what level role simulation will play. I’m confident that it will work for certain questions, but we’ll see how good the models get at predicting human behavior. Konstantine Buhler: I’d imagine it’s actually even more important the better AI gets to have the delta, because the competition—if companies are about serving people—which I think we can all agree on, at the end of the day every company is about serving humans. Sonya Huang: Konstantine, our resident humanist. Konstantine Buhler: [laughs] I’m a humanist, absolutely. But if companies are about serving people, because that’s why we’re all working is to help someone else in some way, and intelligence gets better and better and better, and you kind of have what the human wants here and the intelligence is approaching that asymptote, then the delta in that asymptote—which is what is in a human’s mind that isn’t in the AI’s mind—only becomes more important. Alfred Wahlforss: Yeah. And one of the things that we’ve also realized is there’s a lot of talk around what are the moats for these vertical AI companies. Sonya Huang: And what’s your moat? Alfred Wahlforss: What is our moat? Konstantine Buhler: We’ve got network effects and scale economies. Alfred Wahlforss: Yes, we … Konstantine Buhler: Those are nice. Sonya Huang: I feel like we’re on an episode of Acquired right now. Konstantine Buhler: Hey, I’m feeling it. Alfred Wahlforss: [laughs] Konstantine Buhler: I’m feeling it right now. It’s a good book. I recommend it. Seven Powers. Alfred Wahlforss: Yeah, Seven Powers. Love Seven Powers. On the moats, I mean, we have the clear moats, which are the network effects on the panel where you have supply and demand dynamics. We also have the network [inaudible] data moat—as we do more interviews, you get better simulation. And then the product is very sticky, because you have all these interviews in your platform and you don’t want to lose that, you want to track things over time. But even the simplest things, I think, in terms of product advantages, like, one of the first things Bryan Schreier, one of our Sequoia partners, said was that founders want to build something that’s complex, but customers want something that’s stupid simple and it just works. They don’t want to configure their own workflow. They don’t want to sit and build custom software. And just one example of this is creating the interview guide is really difficult. It’s actually an academic subject. And it’s one of the reasons why you have services firms, because they know what methodology to use if you want to understand pricing or brand perception, these kind of things. Konstantine Buhler: You don’t want to lead the witness. Alfred Wahlforss: You don’t want to lead the witness. And it’s really hard to get that right. In the beginning, we just used the vanilla LLM models. And the customers would create the interviews, they would get the data back, and then they’d come back to us really frustrated saying, “What is this? I can’t use this data for anything.” And we took the blame for that. Now we’ve trained it to follow the best practices so that you always get good data out of the interviews. And I think that’s the advantage you have as a vertical AI company, that you can essentially train this agent to follow best practices in the work that you do. Sonya Huang: So I want to go back to the concept of Tide Pods that you had mentioned earlier. I think it’s really interesting. So much of market research as I understand it today is almost more inviting people to pass judgment on ideas that you feed them. But it seems to me that one of the—you know, hallucinations can be a bug, they can also be a feature with generative AI. And do you think we’re going to see user research actually evolve into live product ideation? I could almost imagine AI inventing solutions as customers are going about their interview process, even helping visualize those solutions. Are your customers doing that already, or do you think we’re going to have the moment where AI can create a Tide Pods idea in a market interview anytime soon? Alfred Wahlforss: Yeah, I think that’s really exciting. Today they do that manually, you know, use AI to generate images of different concepts and feed that into the interviews. But I think specifically also with simulation it becomes really powerful. So we now have an MCP as well, so that you can feed that into Claude and then you can tell Claude to run Listen in a loop and then come up with a bunch of ideas for how to market something or different concepts, and then you can have it run like that. Sonya Huang: I’m even thinking in the course of an interview, as somebody’s complaining about, say, the Tide is not very portable, for the AI to be live brainstorming solutions with you, not just listening. Konstantine Buhler: Yeah, this is what it could look like with an image generator, too. That’d be cool, Sonya. Alfred Wahlforss: Yeah, I think it’s a good idea. You should be on our product team. Sonya Huang: [laughs] Awesome. Well, Alfred, we really love what you’re building. Thank you for taking the time to share insights both on the broader market, which I think is just so fascinating, and also what it takes to be building in the application layer right now. We really admire the business that you’ve built, and thank you for your continued partnership. Alfred Wahlforss: Thank you so much. <div class="framer-mcvrsd" id="

Alfred Wahlforss, co-founder and CEO of Listen Labs, is building an AI agent that interviews your customers at a scale no focus group ever could—thousands of voice conversations at once, drawn from an audience of 30 million people. A year after launch, Listen serves hundreds of Fortune 100s to Startups including Microsoft, Google, NBC Universal, P&G, Anthropic, Cursor, and Cognition. Alfred explains the counterintuitive finding underneath it all: people are often more honest with an AI than a human interviewer, opening up to a non-judgmental entity that costs less and never makes them feel rushed. He walks through why interview transcripts—not credit card data or behavioral logs—turn out to be the richest fuel for predicting how customers will behave, how Listen back-tests its simulations to know which questions it can and can't answer, and why 80% of the company's engineering goes into building the right audience. As AGI makes building trivial, Alfred argues the scarce resource becomes knowing what to build. That's the loop Listen wants to own.

本集资源与出处