The gathering trend towards streamlined machine learning at the edge is spurring the wider IoT market, and driving digital transformation across industries at massive scale.
While complex AI algorithms running on power-hungry Cloud servers are hyped as the solution for Industry 4.0 and tomorrow’s smart cities, the real revolution is the Machine Learning (ML) supporting the compact, resource-optimised wireless IoT sensors underpinning the IoT.
The number of IoT connections is set to nearly-triple, from 6.6bn to 23.6bn, over the next five years (to 2026), says global market advisory firm ABI Research. This means a lot of sensors in the field, skimming-off simple environmental metrics and overflowing with data. The challenge for engineers has been to ensure wireless networks continue to cope by designing IoT devices with the hardware and software resources needed to process ever-increasing volumes of information at the edge.
Today, edge processing on wireless sensors is common in millions of indoor and outdoor smart applications, for example, to optimise energy usage, predict hardware maintenance, improve health and safety, and generally deliver a smarter society. Tiny wireless SoCs with their embedded microcontrollers—and optimized to eke out energy reserves—power these applications. And to maximise the SoCs efficacy, the ML software has adapted too.
The power of inference
The result is TinyML. The software’s algorithms use a scaled-down form of ML. It takes advantage of Cloud-based ML training and model synthesis in the same way that conventional ML does, but TinyML is far simpler and quicker to use than traditional techniques that depend on powerful servers training increasingly large and complex models. Moreover, the lightweight models derived using TinyML run very efficiently on the optimised compute resources of the target edge devices.
Although TinyML models feature streamlined software, synthesis demands the power of Cloud computing. The verified firmware is then downloaded to the edge device. As more sensor data becomes available to refine the models, new versions are created in the Cloud before updating the edge-device over the air. The practical use of TinyML models by edge-devices is known as “inference” and it is revolutionising embedded IoT systems. Little surprise, then, that the TinyML market for on-board IoT analytics is set to grow from 15.2m to 2.5bn shipments over the decade; every new connection, says analyst ABI, will potentially benefit from TinyML.
Nordic Semiconductor’s latest Arm-based Systems-on-Chip (SoCs) and Systems-in-Package (SiPs) are a prime foundation for edge devices as the industry’s migrates to smaller, smarter, ubiquitous IoT. The company’s collaboration with California-based Edge Impulse, the leading developer for TinyML, ensures the technology is seamlessly embedded into Nordic’s nRF52 and nRF53 short range SoCs and the nRF91 Series low power cellular IoT solutions. These products are now enabling ML at the very edge, where the data is produced and used to make automated decisions.
The engineering challenge for Nordic and Edge Impulse is tough because the SoCs and SiPs share their resources among several firmware components. But TinyML inference works with even 10-to-20 KB of memory. If 50 KB is available, then, with the latest compiler techniques to compress parsing and analysis tasks, the application set on the hardware can expand to include real-time audio analysis, audio event analysis, and even human keyword spotting.
Abstracting the complexity from ML
The challenge to popularise TinyML is less about the hardware and firmware and more about making it accessible to a wider cohort of developers. (This is a pattern seen before, in the mobile phone sector. A few years ago, ML applications were difficult to implement on smartphones, but then the technology became far more accessible and now ML applications (for example, virtual assistants) are commonplace on the handsets.) Where ML was once a rarefied discipline for data scientists, who could develop complex algorithms while having access to virtually limitless compute power in the Cloud, Edge Impulse is bringing it into the realm of embedded engineers, who leverage their own specialist expertise in how best to use it for edge-processing in optimised devices.
Edge Impulse has created one of the first open source projects for writing inference algorithms into embedded systems, combining with the TensorFlow Lite framework for microcontrollers. All the heavy-lifting has been moved to the Cloud and wound into an integrated development environment (IDE) called Edge Impulse Studio. This IDE can be used by embedded engineers to create TinyML binary files ideally suited to Nordic hardware. In other words, these new software tools abstract away the complexity to help non-expert developers rapidly bring ML to familiar everyday objects. The result for those developers is the ability to go from data capture to data sets without breaking a sweat, and to train, test, and iterate in a virtuous ML cycle. This movement to democratise ML is part of a wider trend towards Software 2.0, where every-‘thing’ becomes smart, as well as connected. The revolution has started already.
– Originally written by Bjørn Kvaale, Product Marketing Engineer, Nordic Semiconductor –