Exploring Physical AI: 5 key takeaways from the latest session at 3EALITY
Physical AI is moving robotics beyond controlled lab environments and into the complexity of the real world. During the latest 3EALITY Session, part of the recurring event series by HTCE’s innovation hub for Spatial Computing, Dr. Alessandro Saccon (Eindhoven University of Technology) took the audience through more than a decade of robotics research, exploring how robots can better perceive, understand and interact with open-world environments.
The session offered a deep dive into the technologies shaping the future of robotic manipulation, including visuotactile perception, physics simulation, real-time digital twins, motion planning, actuator technology and machine learning hardware. Here’s what we learned.
What does it take for robots to operate autonomously in the real world?
Robotic manipulation is progressing fast and there are definitely some low-hanging fruits
Vision- action policies are enabling new forms of robot programming and opening up new applications. It is imperative to keep a close look at these developments. Still, although robots can already perform impressive motion tasks, even with just two arms and minimalistic hands, true adaptive and robust autonomous manipulation in the open world remains a major challenge.
The key question is not just whether a robot can see and move, but whether it can truly perceive and predict the consequences of its actions and intentionally make contact with the surrounding world.
Real-life scenarios reveal the complexity
The robotics RBT section of the TU/e has a long term experience on real-world tasks such as tossing objects, boxing items, and quickly picking heavy items up. These are trivial tasks for humans, but they require a high level of coordination, perception and adaptability from machines. Robots need to understand how objects behave, how physical contact changes a situation and what the consequences of their actions might be.
No single approach will solve manipulation
Offline planning is currently the established approach for fully structured environments, but many high-mix low-volume tasks still require human operators. Learning-based methods such as imitation and reinforcement learning add new possibilities, as robots can act, observe what happens and potentially improve over time. The future will likely come from combining of planning, imitation learning, reinforcement learning. Fast and accurate physics simulation for machine learning is a key element not to forget.
Robots need to see, touch and imagine before they act
A major part of the vision at TU/e is to build machines that can perceive their environment, understand contact and simulate possible outcomes before taking action. This is where, e.g., a real-to-sim and sim-to-real approach becomes important: learning from the real world, imagining future outcomes in the robot “mind” (simulation) and bringing those insights back into physical robotic systems. This is also essential for safety. Robots still have a limited understanding of the consequences of their actions, which can create risks when they are deployed near people. Better perception, prediction and control are needed before robots can safely operate in more open, human environments.
The first opportunities are likely in semi-structured environments
The most promising markets for these autonomous manipulation solutions seem to be sectors such as logistics and manufacturing, and potentially aviation, where tasks and objects are somehow more structured and the value of automation is easier to identify. For other sectors, such as construction or healthcare, this might be harder to address first because of larger variations in the environment in terms of lighting, temperature, weather conditions, and presence of people in close proximity.
A big thank you to Dr. Alessandro Saccon for the inspiring session and for sharing his expertise. And of course, thank you to everyone who joined the event!
About 3EALITY
3EALITY is the High Tech Campus Eindhoven innovation hub for Spatial Computing and Digital Twin technologies.
We’ve entered a three-dimensional evolution of the Internet, in which virtual worlds are added as a layer to our physical world. 3EALITY fosters an ecosystem that drives the development and application of 3D Internet technologies – such as spatial computing, including virtual & augmented reality, digital twins, blockchain (web3) and IoT.
At the hub, we facilitate an ecosystem of companies and individuals by offering workspace, organizing events and matching supply with demand.
