To mimic the human hand is surprisingly difficult.
Giving a presentation at Hardware Pioneers this week, Dr Stephen Ihmels showed a video of a human hand moving in front of a camera. As fingers flexed and wiggled, a robotic hand beside it mirrored the movements almost instantly.
The demonstration looked simple. The engineering challenge behind it was anything but.
To make the system work, the robot had to perceive its environment, process visual data, run AI models, interpret the results, and translate them into precise motor movements in real time. Every layer, from sensors and processors to communications, power management, and control systems, had to work together.
“Physical AI is not just AI compute,” said Ihmels, Head of Segment Marketing and System Architect for Industrial Automation and Robotics at STMicroelectronics. “It is sensors, processing, control, actuation, and power supply working together as a system.”
As Ihmels pointed out, the difficulty is no longer connecting them to the internet. It is making them intelligent, secure, reliable, and efficient enough to function in the real world.
At Hardware Pioneers Max, held this week at London’s ExCeL Centre, conversations repeatedly returned to the same question: what is now holding IoT back?
The answers varied, but four challenges emerged again and again.
1. Managing growing system complexity
According to Ihmels, the rise of physical AI is accelerating the challenge away from connectivity and towards integrating multiple technologies into a single coherent system.
“We are taking AI from the Cloud into the real world,” he said. “That requires us to think about the entire stack.”
Modern connected systems increasingly combine sensors, AI accelerators, communications modules, motor controllers, power electronics, cameras, lidar sensors, and edge processors. The result is what Ihmels described as a “system of systems”.
The robotic hand demonstration offered a glimpse of that complexity. Visual data had to be captured, processed and interpreted before motor commands could be generated and executed in real time. Any delay, bottleneck or failure in the chain would affect performance.
This growing complexity is forcing companies to think less about individual components and more about how entire architectures work together.
As Ihmels put it, success will depend on understanding not only the intelligence inside a system, but also “the application requirements that come with this from a safety, reliability and efficiency perspective”.
2. Running AI without draining batteries
The rush to add AI capabilities to connected devices is also creating a new challenge for developers: power consumption.
Many IoT devices are expected to run for months or years on batteries, yet AI workloads can quickly consume available resources if they are not carefully managed.
Sam Presley, Technical Product Manager at Nordic Semiconductor, believes the answer increasingly lies in running AI directly on devices rather than sending data back and forth to the Cloud.
“We can run machine learning inference on the chip with significantly lower power consumption than running the same kind of machine learning inference models on the CPU,” he said. “Customers are able to achieve around 10 times improvement in power consumption and also speed.”
Presley demonstrated a camera-based system capable of detecting whether a person was present in an image, with processing taking place locally on the device.
“You don’t want to send all of this data to the Cloud,” Presley added. “It’s very costly in terms of compute in the Cloud. There are also security considerations.”
3. Ensuring reliability as devices become autonomous
Moreover, as IoT devices move beyond monitoring and begin interacting with the physical world, reliability becomes increasingly important.
A sensor reporting incorrect temperature data is problematic. An autonomous robot making a wrong decision can be far more serious.
That was a recurring theme for Harrison Parker, Regional Sales Manager at QNX, who argued that many developers underestimate the importance of the underlying operating system.
“In the Linux world, if one issue occurs in the kernel, the whole thing goes down,” he said.
QNX, an alternative operating system to Linux, designed for mission critical robotics, uses a microkernel architecture that isolates different functions from one another. If one component fails, the rest of the system can continue operating.
Parker illustrated the difference using the example of a surgical robot.
“If my screen dies, the keyhole surgery that I’m doing on my patient isn’t in danger because the robot arm has its own space in QNX,” he explained. “But if I was in Linux and my screen died, my robot arm would die.”
The issue becomes increasingly relevant as autonomous systems move from factories into warehouses, hospitals, public spaces, and eventually homes.
Parker believes many developers are beginning to recognise the challenge.
“For the first time this year, when we did our studies, people were saying software is now a limiting factor,” he said. “Hardware was always seen as the limitation. Now people are starting to realise the software has to keep up.”
4. Security is becoming a lifetime responsibility
Cybersecurity has long been a concern for IoT deployments, but regulatory changes are raising expectations.
Increasingly, manufacturers are being required not only to secure devices at launch, but also to maintain them throughout their operational lives.
“Security is a really important topic across all of our products,” said Presley. “There’s a lot of regulation in Europe, in the US, all over the world.”
The challenge is particularly acute because many IoT deployments remain operational for years, sometimes decades.
“With the new requirements, you’ll need to deploy over-the-air firmware updates for any security vulnerabilities that may be found in the future,” Presley said.
There’s plenty of other editorial on our sister site, Electronic Specifier! Or you can always join in the conversation by commenting below or visiting our LinkedIn page.
