No. 112·中文精读

从4000小时到数百万小时:给人类装传感器,喂饱机器人基础模型

Inside the Robot Data Factory: Mecka's Josh Gao on Sensorizing Humans to Train Robots

Mecka AI 创始人兼 CEO 高乔(Josh Gao)讲述如何用定制传感帽、触觉手套和 100Gbps 上传管线,把人类动作制成机器人训练数据。

节目音频00:00 / 45:11
中文精读由 AI 基于原始访谈改写,非官方翻译,可能存在偏差或错误;表述以原节目为准。

两年前,机器人研究圈最大的第一人称视频数据集叫Ego4D,全部内容加起来约4000小时,拍的是普通人打保龄球、过日常。当时,这已经是整个行业的天花板。

今天,一家叫Mecka的公司,后台上百PB的数据在管道里跑。为了把数据传回来,他们在运营的多个地区专门铺了百Gb级的上传专线。

创始人乔什·高(Josh Gao)最近上了红杉资本的播客Training Data。他做的事说出来有点怪:给活人装传感器,让普通人戴着摄像头和触觉手套过日子,把人类的一举一动变成机器人的训练数据。他管这叫“传感器化人类”。做饭、打扫、拧瓶盖,人怎么干,机器人就学什么。

主持人开场就把位置说清楚了:整个生态都相信机器人要进黄金年代,可机器人被数据卡着脖子。LLM当年吃的是互联网白送的几万亿token,机器人每一秒数据都得花钱雇人干出来。Mecka站的位置,就是这门生意的上游水龙头。

机器人没有互联网可以白嫖

乔什把机器人数据拆成三样东西:像素、动作、逻辑。互联网上有海量像素,动作和逻辑几乎没有。没有人把“拧瓶盖要用几牛顿、手腕先转还是手肘先转”写成文字挂在网页上。

更结构性的一点:LLM的预训练数据当年是实验室自己爬的,基本没形成市场。机器人这边,预训练数据从第一天起就是真金白银的大市场。数据采集本身成了一门独立的、按小时计费的生意,这在AI史上是头一回。

主持人问得很直接:没有互联网这份礼物,靠付费和运营硬采,经济上可行吗?乔什的答案是,起点没那么惨,我们并没有从零开始,互联网时代的遗产还在。视频生成模型和世界模型攒下的功夫都能复用,Mecka也在教视频模型生成更符合物理规律的画面。真正的问题是数据规模能不能滚起来,而每一档数据量对应能解锁什么商业用例。先解锁简单的抓放任务,再往长尾走,长尾比今天大几个数量级。

顺带说一句,Mecka官网上写着“教机器人认识碳基生命”。乔什说网站是他自己写的,发布前一晚赶的工。灵感来自科幻:阿西莫夫、《安德的游戏》、《银河系漫游指南》,故事讲到最后,地球本身就是一台巨大的有机计算机,跑了千万年只为算出一个答案。人脑只跑20瓦,还能自我修复。教机器人学会物理世界的过程,本身就是把碳基生命的秘密一点点展示给它看。

为什么把公司押在第一人称数据上

乔什的背景和机器人八竿子打不着。他大学时做云厨房版的Toast(美国那家餐厅SaaS公司),管后厨订单系统,大四翘课去纽约州罗切斯特的乡下给餐馆装自助点餐机,通宵换掉人家的前厅系统。联创更离谱,做过一个稳定币卖给Coinbase,还在包括英雄联盟在内的四款竞技游戏里打进全球前一千名。两个加拿大人,共同的底色是科幻迷,下一个项目想做个“未来感拉满”的东西,选了机器人。主持人去过他们办公室,一进门就是各种机电装置,墙上还挂着川崎的Kaleido人形机器人。

2024年底进场时,他们把能找到的论文读了个遍,联系了几百个业内人,跑遍各家机器人实验室。翻Ego4D论文时的感受很直接:4000小时听着不少,摊到做饭、保洁、娱乐这些日常场景上,每类薄得可怜。当时行业主流的采集方式是遥操作,人遥控机器人做演示。遥操作的问题是人得跟着机器人走,一个操作员对一台机器,怎么算都扩不起来。GPT的故事摆在那,不把数据规模拉上去,一切免谈。

他们的切入点是第一人称数据(egocentric data):人戴设备干活,摄像头拍下第一视角。逻辑很朴素,机器人本来就从第一视角体验世界,那就让它看看人类的第一视角是怎么跟世界互动的,从人类数据到机器人数据的鸿沟里,视觉这一块能补上大半。再加上人形机器人刚起飞,人和人形机器人长得差不多,人完全可以像特斯拉的车队那样,变成一支给模型喂数据的“人类车队”。

最早的试点糙得很:一个手机App,找来一群华人阿姨叔叔,拍自己做饭、打扫。后来试过UMI这类手持采集方案。手机App铺得快,但信息密度不够,一路演化到今天自研的整套硬件。

数据工厂的真实运营:地推、手套和防骗

这期节目信息密度最高的部分,是数据工厂到底怎么转。

采集端两条线。第一条是C端:手机App注册,填职业,护士、厨师、司机,工种本身就是标签,平台推任务流给你,今天打扫卧室、做早餐、收拾残局。轻量,图的是多样性。第二条是B2B地推:面包房、PCB产线,理论上人类全部体力工种都在清单上,一家一家谈。主持人开玩笑问要不要顺便采点播客数据,乔什说播客里手工活不多,先欠着。

真正的门槛在硬件。比如力数据。抓鸡蛋用多大劲、拉开椅子要几牛顿,视频拍不出来。现在的替代方案是腕部相机,靠看手和物体接触的时间去估力,乔什承认这是权宜之计。市面上根本买不到一副能可靠记录日常用力、还能直接产出训练数据集的触觉手套,只能自己造。Mecka为此跑了好几轮硬件迭代,棒球帽上装摄像头和传感器,加一个中央枢纽,连接触觉手套、外接相机这些配件。乔什管这叫“把人Waymo化”,给人类装上自动驾驶测试车那套行头。

物流和后处理是另一座山。硬件要铺进几千个家庭、几千家企业,数据传回来只是开始,后面还有质量门禁和一整套计算机视觉处理,把原始视频变成模型能吃的数据集。数百PB的吞吐,靠的就是前面说的那些上传专线。

质量是个移动靶。业内此前有过大型仓库式遥操作数据工厂的教训:任务重复到一定程度,多样性过饱和,数据收益掉头向下。人类数据的边缘情况更野,上周还成立的质量规则,这周就可能失效,只能靠启发式规则加快速升级的引擎硬扛。

欺诈是被低估的问题。有人在YouTube上找到“麦当劳打工第一视角”的视频,录下来冒充自己的劳动贡献交上来。

「永远不要低估人类搞钱的创造力。」

乔什的对策分三层:建过滤器,亲自到采集现场看活怎么干,再把激励模型设计成让人愿意交真东西。

至于护城河,主持人问了个尖锐问题:竞争对手跑来出更高的价采同样的数据,中间商利润会不会被挤没?乔什的回答是,付给贡献者的钱在端到端成本里算不上大头,贵的是后面的硬件、运营、质量门禁和处理。差异化在于能不能让数据在大规模下可验证地高质量。产品还会分层进化:第一人称数据只是一个SKU,今天采路边中餐馆,明天采米其林餐厅;今天采业余网球选手,明天逼近费德勒。

百万小时之后,还差几个数量级

行业总算进入了百万小时级的时代,对比两年前Ego4D的4000小时是巨大跃升。但乔什拿自动驾驶算了笔账:特斯拉花了十几年,解决的只是两个自由度,加速和转向,几百万辆车在路上跑,攒的视频远超几百万小时。机器人光一只手就几十个自由度,还要加力数据,环境更野。

作为投机器人基础模型的人,主持人看着这些数字也发怵:这么算下去要几十亿、几万亿小时,全世界的GDP填得起吗,应许之地到底能不能到?乔什把问题拆开:机器人化没有“某天解锁”的二元时刻,它是一级一级商业用例往上爬的过程。底部金字塔搭起来后,泛化出现了,scaling law看得见了,行业才敢往上加注;有了部署,就能在真实场景采失败案例和纠正数据。他的原话是,甚至可以就在这间播客录制间里采数据,把这个房间、这个任务的后训练补齐,冲几个9的可靠性。

还有个拧巴的现实:现在最容易采的是家庭数据,家家都能拍,但家庭恰恰是最难部署机器人的场景。评测体系刚开始成熟,核心指标之一是别让机器人压到婴儿。乔什的判断是先做受限的、有SOP可爬坡的任务。叠衣服这个任务他们做了一年半以上,他认为近期就会解决;产线上那些传统机器人搞不定的抓放、包装,一两年内会陆续拿下。

部署顺序和“现代版系统集成商”

乔什说大家低估了机器人形态的多样性。人形机器人会是流行的终局,但今天中国和东南亚的酒店里,机器人已经在做清洁、做配送,外卖到门口后从前台送到房门,这类机器人早已规模部署,只是在北美还没流行开。最先铺开的会是工业清洁机器人这类任务受限的形态,然后一步步往操作类任务爬。

FSD那种飞轮什么时候来?乔什说还得等,路径是先在商业任务上站稳,部署后拿到失败和纠正数据,飞轮才转得起来。到那天Mecka也不会只守着第一人称数据,用他的说法,“整个数据汤”都重要。

「如果世界真的广阔、细节无限丰富,可以教机器人的东西永远不会缺。」

聊到机器人怎么扩散到整个经济,主持人拿LLM世界类比:垂直应用公司、自己做销售的模型公司、帮企业上AI的服务大军。乔什说物理世界会两条腿走路,既有一竿子插到底的垂直公司,从硬件到模型到数据全包,也会有开放模型加千奇百怪硬件形态的生态。Mecka给自己选的位置是现代版的机器人系统集成商:传统集成商装完设备拿薄利,竞争惨烈;新一代要让装出去的机器人越用越能干,上周装的机器人这周就学会几天前还不会的本事,帮客户把业务边界往外推。在具体场景里采域内数据、做后训练、白手套陪企业过渡,都是这套打法的一部分。

给投资人的建议,他站“通用和垂直都投”。垂直里最看好工业机器人,任务受限,有传统机器人基础,比如数据中心维护,毕竟那个环境里不会突然出现一个婴儿。酒店配送也行,前台到房门还带个交接动作。再往远处,是他们公司全会的内部玩笑:哪天Mecka能帮着部署一台采小行星的机器人,这公司就算成了。主持人补了一句Gecko Robotics,用爬壁机器人给军舰、油罐这类基础设施做检测,方向类似。

明天就能用上的三件事

  • 判断一家机器人公司的成色,先看它的数据从哪来。靠遥操作堆小时数的故事,天花板很低;有自研采集硬件、有质量门禁、有防欺诈体系的,才谈得上数据飞轮。
  • 评估AI项目套用“用例阶梯”思维。别问“什么时候全面实现”,问“当前这档数据和能力能解锁哪个付费场景”。特斯拉先跑公路再谈全自动驾驶,机器人先叠衣服再进厨房,是同一种爬法。
  • 盯住力数据和触觉硬件这条暗线。视频解决“看见”,力数据才解决“拿捏”。谁先把触觉数据规模化,谁就握着操作类任务的钥匙。

最后留个乔什式的注脚。人脑20瓦,会自我修复,还能边过日子边攒数据。机器人要追上这个配置,得靠一个行业一秒一秒把人类的日常买下来、洗干净、喂进去。这门生意刚跑到百万小时,离终点还差几个数量级,但水龙头已经拧开了。

文稿来源:Megaphone 音频 + Whisper 转写;机器转写可能有识别错误,人名与数字请以音频为准。
There's an idea in science fiction that is when robotics and AI has ultimately fully generalized and solved all tasks that humans can do, the final frontier is going to be organic. Final frontier is going to be biological. And you see this in like Asimov, you see this in Ender's Game, even in Hitchhikers at the very end. The planet ultimately was this giant organic computer trying to process and understand what the meaning of life was or what the question was. Today, I'm delighted to have Josh from Mecca on the show. Josh is founder and CEO of Mecca AI, which is building a data and deployment company for robotics. I and I think the whole ecosystem think that robotics is about to enter a golden age. And unlike LLMs, where we had so much free human data from the internet, robotics is fundamentally data constrained. And I think your position in the ecosystem as the leading data factory to bring robotic data and physical data in distribution for models is just a really, really interesting perch. So I'm excited to use this conversation today to chat about how does egocentric data and data collection work in robotics? What are the actual operations and economics of operating a data factory? And your overall thoughts on robotics research and deployment. Thanks for joining the show. Amazing. Thanks for having me. Awesome. Maybe to start off, for those who aren't familiar, what is Mecca and how are you looking to impact the world of robotics? So Mecca is, like you said, the data and deployment company for Physically AI. What that means is today we create, we process, and ultimately we license a lot of the data that goes into training these robotics foundation models. The big problem across robotics is that we don't have the luxury of the cold start that LLMs had with the internet. Fundamentally, a lot of this robotics data is three things. It's pixels, it's actions, and it's logic. And so we don't have access to a large amount of this curated and processed and clean data to be able to then train these models to understand what it is to be in the physical world. So a lot of our job is pretty much to, one, create the different types of hardware, such as sensorizing and wearables for humans. The different operations to go and collect and partner with individual contributors, businesses, tasks across different environments. And then ultimately post-process and own the quality gates to make these data sets useful and teach robots about the world. Without having been given the benefits of a giant, free, many trillions of tokens, maybe lots of trillions, trillions of trillions of tokens of internet corpus, do you think that robotics, you know, having to pay for and operationally collect all the data that's going into the mix, is it even a feasible challenge? Like, is it economically feasible without having had the gift of the internet? I think we still have the luxury of a lot of things that was given from the internet. So we aren't starting from zero, ultimately. I think you can see it across LLMs and data across LLMs. Pre-training data was always driven by the labs, largely, across web crawlers, things like that. It didn't become a large market. Today in robotics, we do see pre-training as quite a large market. And that could be taken in several forms. So that could be that we are teaching video models how to generate more physics-grounded realities of what the world looks like. And we get to leverage all the work that has gone into the BLMs and different types of world models before to make that possible. Is it possible and scalable to get to that volume of data? I think that ultimately comes down to what commercial use case can you unlock at which levels of scales of data? And our strong belief is along that journey that we will be able to unlock a lot of the simple pick-and-place tasks along the beginning of what we're creating. And towards that long tail, that is going to be many orders of magnitudes larger than it is today. I did a little homework for this. Your website is very whimsical. It says that you are teaching robots about carbon life. What does that mean? I actually made the website. I love it. Like the night before we did our first tweet announcement or something like that. There's an idea in science fiction that is when robotics and AI has ultimately fully generalized and solved all tasks that humans can do. The final frontier is going to be organic. Final frontier is going to be biological. And you see this in Asimov. You see this in Ender's Game. You could even in Hitchhikers at the very end. The planet ultimately was this giant organic computer trying to process and understand what the meaning of life was or what the question was. And so I think in the same vein for us, that is the human brain runs on just 20 watts. We are self-healing. And so along the journey of teaching and getting robots to fully generalize, we'll slowly show them what carbon life is made of. Love that. Okay. You come from an untraditional background for robotics. I think you started in the fintech world and the sneaker world, if I'm not mistaken. Tell me about how you and your co-founder found your way to robotics. It was quite an interesting journey. Yeah. So before this, I was building essentially a toast for cloud kitchens. This meant we did everything from back of the house order management. At one point in my fourth year of university, I was skipping class to go deploy and install kiosk into rural Rochester, New York, where we had to stay all nights replacing a lot of these different front of the houses for restaurants. My co-founder built a stable coin. They ultimately sold to Coinbase, but he's been this very obsessive personality. He was a top thousand in four competitive games, including League of Legends. And I think a big part of what brought us together, one, we're both Canadian. And two is we have this big science fiction grounding. And so the belief ultimately was that payment was a hard and difficult problem. But whatever we worked on next, we wanted to be something that seemed exciting that the future was going to be incredible. And robotics seemed like a great space to get into. I remember walking into your office and then immediately seeing those like mechatronic robots and then the Kawasaki Corleos on the walleye. The inner child in you must be very happy about what you're doing these days. Yeah, absolutely. Coming from a non-robotics background, what do you think your advantages are in building this business? And where has it presented challenges? I think the advantages is when we entered the space, we had to go and do this full discovery process. We reached out to hundreds of folks that were in the space that we appreciated. We read every paper that we get our hands on. We went to visit the different robot labs. And what we saw in late 24, roughly, GPT was taking off. Robotics was still evolving from this traditional robotics era. And it seems obvious to us at the very least that data and compute would become very important across the industry. And so moving towards that, we got to read and see some of the papers like Ego4D from a fresh lens and say, hey, there's about 4,000 hours of this human demonstration data. And when you actually went in to look at the data, you would realize some of the data sets would be daily life of folks like bowling, for example. There was very broad use cases and potential applications at the 4,000 hour scale. And so coming into that with a fresh lens, we were able to, from the first principles approach, I'd say, have a general intuition of what would be important and then iterate very closely with people that were much smarter than us that had different views of what the future would be and be able to stay close to iterate on what the product would become. Yeah. Interesting. It seems like robotics has just changed so quickly as a field that almost coming from the field can be an advantage and a disadvantage because people come from different eras of, you know, what's important in the training mix and things like that. Yeah. Maybe for our audience, just say a word on like, what is egocentric data? And I think you really started your company kind of making a bet behind it. Obviously, it's much more distributed now, but what is egocentric data? Why did you decide to bet the company on it at the beginning? Egocentric data is pretty much video data from humans demonstrating and doing tasks from a first-person POV. The high-level idea was that from the gap of human data to robotics data, you could close a lot of the vision gap. Where robotics experience the world from POV, can we also show it what it is like to interact with the world from first-person as well? Why we bet on it was pretty much when we explored the space. A lot of what we saw was robotics data at the time was teleoperation, which was humans controlling robots to do demonstrations. It was very hard to scale at the time. And from what we saw across GPT and a lot of the principles of scaling up, we had to find ways to scale up data much more significantly. And I think some of our early intuitions were that humans and robots share a lot of similarities. Humanoids were just starting to take off. Humans and humanoids look the same. Why couldn't we turn humans into this fleet-style learning that Tesla has to be able to all contribute into understanding and ground the physics of the world? And so what does that look like? You're like mounting an iPhone on somebody's forehead and just letting them go about their day? Or what does that look like in practice? Yeah, when we first started, our very earliest pilot was we created a mobile app. We got all the Chinese aunties and uncles to record themselves cooking, cleaning, doing tasks. That obviously evolved a lot. We tried a number of things. We did UMI data. We did all the sorts at the beginning. And that's evolved quite a bit into all the custom hardware that we have to produce now and all the different partnerships that we have to have across different industries. And ultimately, we're sensorizing humans. And that's the broader category. How important is it to get force data from the sensors? Because one of the things I've always thought about is in self-driving, you don't really need force data as much, right? Whereas with robotics, especially with manipulation, you really care about how hard am I squeezing that door handle or picking up that egg? And it seems to me that video doesn't quite capture that. You might be able to approximate it, but it doesn't really capture it. How important is capturing force data? And how do you think about getting that? I think some of the substitutes for force data that we've seen would be approaches like wrist cameras currently across both humans and robots, and being able to see the contact time between the hand and the different objects that you interact with. That obviously is kind of a hacky way, ultimately, to get to understanding how much force in newtons it takes to pull open a chair or twist a bottle cap. I think force data will be incredibly important. It's actually one of the products that we've been building out in that when you iterate with many different robotics foundation labs, you realize to close this gap, one, we have to create new hardware. So we've been developing things like gloves for humans to wear. And in those gloves, being able to collect the information and how much force it takes to do your daily life and different tasks. I'd love to just spend a bit of time on the actual operations of a data factory, because I think there's so much money that's gone into the LLM data factories. There's so much money that is starting to be poured into and I think will be poured into the robot data factories. And the way they work is just mysterious and intriguing to me. So I'd love to ask you a bit about the operations of it, if that's okay. Yeah, let's do it. Okay. So how does it work? Like if I am, you know, if I want to make an extra buck, do I just sign up on your website to start, you know, to put a camera on my, on my forehead and start sending you data or how does it work? So we actually have a lot of different approaches on the operator side. One would be, we have a direct consumer approach. And so through direct mobile apps, can you sign up from anywhere in the world? Can you start contributing? Can you, can you register which profession you're in? And can you get streamed a sequence of tasks to do such as cleaning your bedroom, cooking your breakfast, cleaning up afterwards? This is a very lightweight approach to get that diversity ultimately of this pre-training data. The second approach that you have to get into is ultimately there is so much more beyond the home to collect. And so we had to build out a lot of this B2B engine and this muscle of being able to recruit businesses across this incredible taxonomy of all physical human work, which is across bakeries. It could be a PCB manufacturing line, everything to... Podcasting. Podcasting. There's not too much handwork in podcasting just yet. Trying to make a quick buck here. We should have equipped ourselves. And that's ultimately how contributors come into the mix. But the actual data engine for the data factory is so much more sophisticated and much more painful than folks can expect. And that is that a lot of the things that we have to distribute into the world didn't exist until we designed and produced this hardware. Even across tactile gloves is a fantastic example. There is nothing off the shelf that you can purchase and that you can wear and that reliably tells you how much force you are applying day to day and turn that into ready data sets for these different robotics foundation models. So we have to have that hardware iteration loop to have these three, four different product cycles, that engineering muscle to actually think and iterate what is going to be useful and then scale that production into the world. And that becomes this logistics of distributing into these thousands of homes, thousands of businesses to collect that, to upload that. We've had to set up these hundred gigabyte upload lines in a lot of the places that we operate in just to sustain the hundreds of petabytes of data that we have to deal with at this point. And then you have to process, you have to make sure your quality gates are incredible, and then all of the computer vision to make it useful ultimately. So let's say you decide, okay, in my training mix, there's not enough podcasters. There's not enough, you know, I don't know, donut bakers. Do you go out and literally go knock on doors of donut bakers then to sign them up to the platform or how does that work? Well, we'll generally track kind of like a heat map across different jobs, industries, and ways we think about diversity. And across that heat map, if ultimately we think, oh, not enough Sonia's podcasting and we have to go get that, that's actually why we're here today. This is our secret message. That's why there's cameras everywhere. No, you're never automating me out of this. No, I'm kidding. Please do. Okay. Interesting. Like I've heard that, you know, there's a lot of quality issues in robot data collection generally, and that a lot of the hours of robot data sample that get sent in end up not being accepted. Why is that? And what are you doing about that problem? This lesson came from even originally the different types of robot data. There have been these very large scale robot factories that have spun up from just large scale warehouses, repetitively doing tasks, trying to scale up teleoperation data. Some of the learnings there came from the research in that there is some threshold of diversity where you oversaturate how much robot data you need or how much data in general you need. Across human data, quality, you can imagine, is actually quite a bit of a moving target as we grow. The amount of edge cases that you have in the real world that you have to deal with are very extreme. And you will always see something new that you haven't seen. So for us, it's been about, can you think about the heuristics to be able to catch a lot of the different things as it's processed? Can you have the different engines to escalate very quickly? And ultimately, we will learn new things about quality and diversity that weren't true a week ago, even. And so the nimbleness to move very quickly to create and adapt your engine to catch that. What's an example of something that you would catch? Well, I think fraud is an underestimated one across data industries in general. We're ultimately creating data that doesn't exist yet versus when you're filtering through the internet. And so when you are, you're coming across very creative and interesting. Like people are sending generated videos or what? Yeah, very creative people where I think at one point we have folks recording YouTube videos like POV working at McDonald's and trying to submit that as contribution pretty much. Wow. Never underestimate how entrepreneurial people can be. That's amazing. Yeah, very much. And so these have been like part of the painful lessons where, yes, you have to create a lot of the gates, but ultimately you have to also be there where collection is happening. You have to go see how it's done. You have to figure out an incentive model that actually incentivizes people to create good contribution. How do you think about your right to earn a margin long term? And, you know, I can imagine a Mecca competitor comes to Sonya and says, Sonya, I really want your podcast data. I could imagine that, you know, the margin that you're able to own as an intermediary ends up getting squeezed. I think the contributor amount in terms of the margin cost is actually not the largest factor at play here in terms of the end-to-end making sure this data is useful. So even though that baseline is always going to be there and maybe it becomes more and more competitive, there will be ways to differentiate and make sure that your data is verifiably high quality at very large scale with the right hardware, the operations, and the software. I think when we think about these products, ultimately it is that we do want to scale up and mature these products. And just like any product that begins to scale up, you want to optimize it. But over time, you may have different products that evolve. And so when you think about this egocentric data of podcasting as simply one of your SKUs, and you then realize that there is going to be different types of data that are going to be important, such as maybe we move from the Chinese restaurant from the side of the road. Can we start moving into the Michelin star restaurants over time? Can you get more and more closer into what it means to be Federer than it means to be an average tennis player? So these products will change and how these products are created will also change. Really interesting. What scale of data has already been collected, do you think? And how much data do you think is going to be collected over the next few years? I think we finally entered a world where we are in millions of hours of data. And coming from the world where Ego4D of 4,000 hours two years ago, where we were then, is a huge step up. But then comparatively, and you think about autonomous driving as an example, Tesla spent over a decade solving for two degrees of freedom, ultimately, of acceleration and direction, millions of cars on the road. They have a lot more than a few million hours of video data at this point. But so relatively, we're still many orders of magnitudes away from where we have to be, ultimately. So on one hand, that sounds like opportunity for you. That sounds like a lot of revenue for the robot data factories. On the other hand, as somebody that invests in the robot foundation models, I look at that and I say like, oh, shoot, are we ever going to get to the promised land? Because I've done the same math as you on like a car has two or three degrees of freedom. A robot has- Even dozens in the hand. Yeah, exactly. And so you combine that with like, you probably need forced data. And there's a way more, you know, it's just a way harder environment to operate in. And I get to like, you might need billions, trillions, quadrillions of hours. Is there even enough GDP in the world to support the cost to bring that in? Like, what do you think? I think it's about unlocking commercial use cases as we go. So it doesn't have to be so much binary in that, oh, one day we've unlocked robotics or not. It is going to be a slow creep up into the world. A great ways to get there is that with some of this base pyramid built up where we are seeing the generalizations, we are seeing the scaling laws at play. That gives people the courage to scale up. And with that courage, can we one, have some generalizability in some of these environments? Can we then also collect a lot of the in-domain, in-task data that closes that gap in post-training? So maybe I can collect data exactly from this room, exactly in this podcast demonstration to actually close the gap of what is it in a lot of cases for nines of reliability in these in these real world environments. That's a helpful framework. So it's not about a binary of getting from zero to one. It's about we're going to increasingly get, you know, robots that can do an increasing set of capabilities. And we will unlock a lot of tasks along the way. Yeah. One curiosity, though, is that, you know, the data that is most plentiful right now is like it seems like home data, right? Just because like everyone, everyone can film that. It's pretty easy to get. But a robot in the home almost seems like one of the most difficult environments to deploy in. And so I'm curious what applications you think will hit first in robotics, because like where the data is most readily available seems almost the opposite of where robotics might be easiest to deploy. The home is going to be some of the environments that we have to be most diligent on by far. And a lot of these benchmarking and evals that have just started to mature across robotics, making sure that the robot doesn't fall on the baby in the homes, for example. I think the initial tasks will be much more constrained as we go along that spectrum in that very goal oriented where you can hill climb some of these tasks, where you have existing SOPs for what it means to do that job and that work. Like a laundry folding robot. It could be laundry folding. Laundry folding. We spent about a year and a half plus trying to solve. And I think it will absolutely get solved in the near term. A lot of these pick and place, these packaging tasks that you see on assembling lines that had to be manipulation bots that couldn't be solved by traditional robotics. We will get there within the next year or two. It is a good transition to talk about deployment. I'd love to talk to you about, you know, robot deployment overall as a concept. You've said that useful robots can already be deployed today. Where do you think that's the case? I think people underestimate how diverse robotics can be. And if we constrain ourselves to, one, humanoids are incredibly popular, it will be a very popular terminal state of robotics as well. But beyond that, there are so many different form factors that exist in robotics today. You could look at different markets, even across hotels in China or Southeast Asia, for example, where there are robots that are doing services, that are doing cleaning, that are delivering from after your Uber Eats gets to the door, from the front desk to your hotel room door that exists. Many of these bots are widely deployed across the world today. They are just starting to get popularized across the U.S. and North America. I think some of the first bots that have to get deployed will be even just the industrial cleaning bots, for example. A lot of like thematic robotics that are out there. And then slowly stepping up and up from there, where you start to solve early manipulation tasks along that spectrum, all the way up until we get to that humanoid solving out of the box, any task in the future. You mentioned Tesla and FSD earlier in this show. When do you think we're going to get to the point where there is a little bit of an FSD-like flywheel? And when we get to that point, how do you see your role as a company evolving? When we will get there, I think we're a while away. But it'll be a slow transition in that when you are slowly solving some of the commercial tasks, you will also be able to finally get some of these flywheels. Then you can get some of the in-deployment data where you get the failure cases, you get the correction data. That will be absolutely important in starting up this robot flywheel. How far that is depends on that spectrum and across saturating different types of tasks and domains that aren't completed or solved for yet. So it's quite a while away. When we do get there, the thing is, I believe that Mecca will look very different along that process. In that one, we're not fully subscribed to only egocentric data. We believe generally that a lot of different SKUs, the entire data soup is important. And so will we be producing the Michelin star data at some point in the future? Will we be teaching some sort of edge case about the real world that the robots haven't gotten to yet? If we believe that the world is incredibly vast and rich in detail, there's going to be no shortage of things to teach. And then even in that world, we do have this angle where we believe that Mecca will very much contribute to the deployment of robots. And so in some of these environments, in closing that gap of being able to do that task in that domain, in that environment, can we actually be the platform where we collect that domain data, we post train, we help support the enterprises and white glove hold them into what it means to be in a world filled with robotics? We see Mecca maturing into that. I'd love to take your predictions on how you think deployment, diffusion is going to work out in the physical world. And maybe if I can analogize what's happened in the LLM world, there's like a few different ways that deploy codes have happened, right? There's vertical specific application companies. There's the foundation model companies themselves that are scaling up GoToMarket. And then there's, you know, all these FDE, deploy code, service code, whatever you want to call them, you know, just armies of people helping enterprises adopt AI at scale. Those seem like the broad vectors that are disseminating tokens across the economy. What do you think the analog will be in the physical world? We believe very much in both the vertical and the horizontal across robotics. So the full verticals that are very task specific all the way from hardware to models to data, the tackling. We also believe in an ecosystem where there's going to be very vast in different shapes and sizes of hardware of robotics. And there will be different types of open models that exist in the world. I think it will look something like a modern robotics version of a systems integrator. That in a traditional version, you are installing the product. It's incredibly competitive. It's tight margins. There is now a world where you can actually install and help make that product become more data-driven and productive over time. So can you actually have the bot you installed last week do something that you didn't think was possible a few days ago even? Can it help you start expanding the scope of what your business is capable of doing? So that's ultimately the milestone that we're hoping to aim for. Okay. But you would tell me as an investor. Now give me some investing advice. You'd invest in both the general and the vertical robots then. Yeah. What are your favorite vertical robot categories? I think the best ones will be the industrial bots to start. Those are going to be ones that already have a lot of traditional robotics in the mix. They're task constrained like we talked about. Like data center maintenance robots? Yeah. You just don't have to expect a baby to appear and have to think about that. And not fall on it, for example. So the world is much more constrained than in the wild. Other categories we're excited about are just spaces that are even feasible and possible early on. And so some of the approaches in the hotels, for example, where you could actually start delivering from the front desk to the hotel rooms, be able to actually have some manipulation to hand off, are exciting to us. And then places that humans were not capable of. A joke that we make in some of our all hands in the company would be if one day Mecca was able to help deploy a robot that mines an asteroid, we've probably done a pretty good job. It's just, it's a job and a task that completely doesn't exist, that humans can't do, that we have no idea how to do just yet. But it seems inevitable. Yeah. I love that. Reminds me of, you know, even, I'm not thinking interplanetary yet, but, you know, like, have you heard of Gecko Robotics? What they do to help maintain chips and things like that. I think it's really cool. Absolutely. Can you say a word on, and whatever you're willing to share publicly, the slate of hardware that you've developed and where the labs currently are most interested in collecting bespoke data? Some of the iteration came from when we started with the mobile app. We quickly were able to scale that. There were a lot of virtues in that product. The downsides, ultimately, were you don't get as much rich information about the world as you could have. And so some of the things that we had to develop very early on were things like Waymo-fying people, ultimately. And in this sensorization, can you build things like these baseball caps that are equipped with different cameras and sensors? We build a central hub that then connects to all these different accessories, such as the tactile gloves, external cameras, and more. All of which is trying to capture how do you interact with the world on a day-to-day and hopefully be able to clone that behavior. Interesting. What is Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? The other introductions that I hear most frequently to the types of data being collected are, you know, you could watch millions of hours of tennis data and you still wouldn't know how to beat Roger Federer, right? And I'm curious what you think of that and what it means for where we might be headed in terms of the way robots learn. Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse? Egoverse?

Robotics never got the internet's gift of free training data, so someone has to manufacture it. Josh Gao, founder and CEO of Mecka AI, runs one of the largest robot data factories, and he explains what that actually involves: custom baseball caps and tactile gloves because nothing off the shelf captures force, 100-gigabit upload lines to move hundreds of petabytes, and custom quality gates. Josh lays out the training recipe the industry is converging on (roughly a million hours of egocentric human data, tens of thousands of hours of robot data, and a few hours of in-domain teleop ) and why tactile and simulation data still have to enter the "data pot of soup." He argues the field is still orders of magnitude behind Tesla's FSD, which spent a decade on two degrees of freedom while a robot hand has dozens. His take: there won't be a single GPT-3 moment for robotics, but a steady unlock of commercial tasks, starting with constrained industrial work and ending, he hopes, with robots mining asteroids.

Hosted by Sonya Huang, Sequoia Capital

0:00 – Introduction

1:46 – What Mecka is: pixels, actions and logic

2:45 – Is robot data economically feasible without the internet?

4:15 – Teaching robots about carbon life

5:27 – Cloud kitchens, stablecoins and the path to robotics

7:03 – Entering robotics with a fresh lens

8:45 – What egocentric data is and why Mecka bet on it

10:09 – The first pilot: aunties and uncles with a mobile app

10:42 – Force data and tactile gloves

11:55 – Inside the operations of a data factory

14:41 – Heat-mapping the taxonomy of human work

15:42 – Quality gates, fraud and the McDonald’s POV videos

17:44 – Earning a margin as a data intermediary

19:33 – Millions of hours vs. Tesla’s decade

20:55 – Why it isn’t binary: unlocking commercial tasks along the way

22:02 – Home data is plentiful; home robots are the hardest

23:24 – Useful robots can be deployed today

24:35 – When robotics gets an FSD-style flywheel

26:42 – Deployment diffusion and the modern systems integrator

28:21 – Favorite vertical robot categories; mining an asteroid

30:08 – The hardware slate: Waymo-fying people

30:55 – EgoVerse

32:04 – The data pot of soup: the converged training mix

33:54 – Move 37 for robotics: AlphaGo and Federer

36:02 – World models and the training mix

36:58 – Will scaling the current mix get us to the promised land?

39:05 – Will the AGI labs steamroll robotics?

40:50 – Sci-fi roots and the Kawasaki Corleo

42:44 – How many robots in a decade?

本集资源与出处