u/Responsible-Grass452

Matthew Johnson-Roberson, Dean of the College of Connected Computing at Vanderbilt University and former director of the Robotics Institute at Carnegie Mellon, argues that physical AI may not follow the same path as large language models.

Language models had a clear training target: predict the next word. That gave researchers a simple objective that could be scaled across massive amounts of text.

Robotics does not appear to have the same equivalent yet.

A robot can collect large amounts of video, sensor and encoder data, but that does not automatically solve the harder problem: what should the system actually optimize for?

Predicting the next frame, joint angle or robot motion is not as universal as predicting the next word in a sentence.

Sergey Levine describes physical AI as something broader than humanoid robots doing household chores.

People often imagine the future of home robotics as a general-purpose robot shaped like a person. Levine gives a different framing. Once computers had general-purpose operating systems, they did not all take the same form. They became desktops, phones, cars, appliances, embedded systems, and many other devices shaped around specific uses.

The same logic can apply to physical AI in the home.

A laundry robot, a dishwashing robot, a mobility assistance robot, a cleaning system, and a home monitoring device may not need the same body. Some may move. Some may be fixed. Some may be built into appliances or furniture. Some may not look like robots at all.

Sergey Levine frames today’s AI systems as being trained mostly on records of human experience.

Text, images, video, code, and other internet-scale data are powerful, but they are still indirect. They come from people who already understand gravity, friction, tools, failure, cause and effect, and the basic rules of physical environments.

Physical AI could add a different kind of training signal. A robot operating in the real world does not only observe outcomes. It takes actions and gathers data from what actually happens next. Objects slip. Plans fail. Tools behave differently than expected. The environment pushes back.

Levine’s argument is that large-scale physical systems could eventually produce data that matters beyond robotics. That data could help models learn more about causality, intuitive physics, object interaction, long-horizon behavior, and recovery when a plan breaks down.

Sergey Levine describes a robotics project where his team contacted 33 research labs and asked them to share data from their own robot setups.

Each lab had different robots and different tasks. Some were working on cable routing, while others were working on taking out the trash or putting objects into drawers.

His team trained one model across all of that data and sent it back to some of the labs to compare against the systems those labs had built for their own tasks.

According to Levine, the generalist model performed about 50% better on average than the lab-specific systems.

Zach Jackowski, GM of Atlas at Boston Dynamics, talks about how getting humanoids into real environments matters, but running the same behavior at scale is not enough. If a fleet is only doing automotive part sequencing, the resulting dataset will mostly improve performance on that task family. It does not automatically produce broad manipulation generalization.

That is why he pushes back on the idea that the path is simply “deploy robots, collect lots of data, and generalization follows.” The harder part is collecting varied, useful data while still operating in controlled enough settings to make the robot commercially useful.

Colin Angle, Roomba co-founder and former iRobot CEO, has launched a new company called Familiar Machines & Magic focused on home robotics.

His view is that humanoids are not the obvious starting point for robots in the home. A home robot should be designed around the job it is meant to do, not around copying the human body. A $20,000 humanoid pushing an upright vacuum is not a practical use case when robot vacuums already exist.

For home robotics, Angle points toward robots built around routine, interaction, wellness, and companionship rather than general-purpose humanoids trying to handle household chores.

Researchers at EPFL developed a control framework that allows robots with different mechanical designs to perform the same task without rewriting code.

The method captures human-demonstrated actions and converts them into a general motion strategy based on kinematics. Each robot then adapts that strategy to its own joint limits and structure, rather than relying on retraining or large datasets.

In testing, different robots completed parts of the same assembly sequence using the same learned task. Each executed it differently, but within safe operating limits.

The goal is to reduce the need to reprogram tasks when robots are replaced or systems change, while keeping behavior predictable and consistent.

Short-range autonomy in industrial settings gets framed differently than highway driving. The focus is on repeatable routes, low speeds, and environments where conditions stay relatively consistent.

More of the challenge sits in near-field perception and precision. Tight spaces, docking, and constant start-stop movement leave less room for error, so coverage and control matter more than long-distance sensing.

Autonomy also ties into a larger sequence of actions. Movement is just one step alongside loading and unloading, so timing and coordination become part of the system.

It leans into a pattern where constraints make the problem more tractable early on, with reliability taking priority before expanding into less predictable environments.

Carnegie Mellon’s Martial Hebert explains that the underlying technology for self-driving cars has been in place for some time, but deployment depends on the conditions the system is operating in.

Driving in heavily mapped, controlled environments with known variables is very different from operating in areas that haven’t been seen before, with changing conditions, varying pedestrian density, and unexpected scenarios.

Each of those factors can require different approaches in sensing, training, and system design. On top of that, systems have to go through extensive testing and validation before they can be used around the general public.

The gap between something that works technically and something that can be validated for real-world use is where most of the time has gone.

Professor Ranjay Krishna explains a gap between modern AI and robotics.

Language models can take examples, adapt to new inputs, and improve output in real time. That behavior does not translate to physical systems.

In robotics, if a task changes even slightly, the system often fails. A different object, a new position, or a small variation in the environment can break what it learned.

The idea of showing a robot how to do something once and having it learn by watching is still out of reach. Research areas like imitation learning and continual learning have not solved this in real-world settings.