For many years, embedded system design followed a relatively stable pattern. Engineers selected hardware, chose an operating system, and built application logic on top. Systems were typically self-contained, and architectural decisions could be made within clearly defined boundaries.
As connectivity became standard and IoT adoption increased, that model began to break down. The industry shifted towards discussions framed around Edge versus Cloud, but this reframing often obscured a more fundamental issue.
In most cases, the challenge is not where compute takes place, but how systems are designed across Edge, Cloud, data, and lifecycle constraints as a unified architecture.
Where IoT systems go wrong
Across industrial and embedded deployments, a consistent pattern has emerged. While the value of connected systems and operational data is widely recognised, turning those capabilities into reliable, scalable, and maintainable systems remains difficult in practice.
The root cause is rarely a lack of technology. Instead, system components are frequently designed in isolation. Hardware, software, connectivity, and security decisions are made independently, often by different teams with different priorities and constraints.
Individually, these components may function correctly. However, when integrated into a full system, weaknesses appear. These typically relate not to performance of individual elements, but to gaps between them, particularly around data flow, timing, lifecycle management, and operational dependencies.
As systems scale and move from pilot deployments into production environments, these gaps become more visible. What initially appears to be a collection of manageable design decisions becomes a system-level coordination problem.
Most IoT failures are therefore not caused by missing capabilities, but by architectural fragmentation introduced early in the design process.
Rethinking the Edge versus Cloud divide
IoT systems are still often described in terms of a trade-off between Edge and Cloud. This framing suggests a binary decision, but real-world systems do not operate in this way.
In practice, IoT architectures span both domains, with each layer serving distinct and complementary roles.
Edge systems are typically responsible for time-sensitive processing, deterministic control, and operation in environments where connectivity may be intermittent or constrained. Cloud systems, on the other hand, provide aggregation, long-term storage, fleet-level visibility, analytics, and orchestration.
In industrial environments, this division is particularly clear. Control systems in manufacturing or process automation must operate locally to meet strict timing requirements. At the same time, the data generated by these systems is valuable beyond the local context, supporting optimisation, predictive maintenance, and cross-site coordination.
The architectural challenge is therefore not choosing between Edge and Cloud, but defining how functionality is distributed between them. This includes decisions about what data is processed locally, what is transmitted, and what is acted upon at different layers of the system.
These decisions are foundational. Once implemented, they are difficult to change without significant redesign, particularly in systems expected to operate continuously over long lifecycles.
AI as a stress test for IoT architecture
The growing use of AI in industrial and embedded systems is increasing pressure on existing IoT architectures. While AI is often viewed as an application layer capability, it is increasingly dependent on how systems are structured at a lower level.
AI systems rely on data that is collected, processed, and interpreted across distributed environments. This introduces dependencies between edge devices, network infrastructure, and cloud services, each with different performance and reliability characteristics.
In well-structured systems, data pipelines are designed with these constraints in mind. In less mature architectures, data often exists in large volumes but is inconsistently structured, inconsistently timed, or difficult to access across system boundaries.
This leads to a common issue: organisations may have access to significant data resources, but struggle to operationalise them effectively. The limiting factor is not data availability, but architectural coherence.
In this context, AI does not introduce a new category of problem. Instead, it exposes limitations that were already present, particularly in how systems manage data across distributed environments.
From component thinking to system thinking
A key reason IoT systems fail in practice is that they are often designed as collections of components rather than as complete systems.
In a component-driven approach, individual teams optimise for local performance, local constraints, or local delivery timelines. While this can improve efficiency at a team level, it often leads to misalignment across the broader architecture.
A system-driven approach requires a different perspective. Data flows, processing responsibilities, connectivity assumptions, security models, and lifecycle management must all be considered together, rather than as independent design domains.
The challenge is that many of these decisions are made early in a project, often before full system requirements are understood. As a result, misalignment is frequently embedded into the architecture from the outset and becomes increasingly costly to address later.
This is particularly relevant in industrial environments, where systems are expected to operate for many years, evolve over time, and remain supportable across multiple hardware and software generations.
Data flow as an architectural decision
One of the most important, and often underestimated, aspects of IoT system design is data flow architecture.
Decisions about where data is generated, where it is processed, how it is transmitted, and where it is stored have direct implications for latency, cost, scalability, and resilience.
These decisions also determine how effectively systems can support higher-level capabilities such as analytics and automation.
For example, processing data locally at the Edge can reduce latency and improve resilience in environments with unreliable connectivity. However, centralising processing in the Cloud can improve consistency, simplify updates, and enable broader system-level analysis.
In practice, most systems require a hybrid approach. The key challenge is ensuring that this hybrid model is designed deliberately, rather than emerging as a result of incremental decisions made in isolation.
Once deployed, changes to data flow architecture can be difficult, particularly when they involve rebalancing processing responsibilities across distributed systems.
Lifecycle management and long-term system behaviour
IoT systems are increasingly expected to operate over long lifecycles, often spanning multiple years or even decades. This introduces additional complexity that goes beyond initial deployment.
Device management, software updates, security patching, and configuration control all become ongoing concerns. If these elements are not designed into the system from the beginning, operational overhead increases significantly over time.
Lifecycle management is therefore not a separate function, but part of the core system architecture. It influences how devices are provisioned, how software evolves, and how security is maintained throughout the system’s operational life.
In industrial environments, this is particularly important, as downtime, manual intervention, and large-scale device replacement can be costly and disruptive.
Security as a system property
Security in IoT systems is often treated as an additional layer, rather than a fundamental design constraint. This approach is increasingly difficult to sustain.
Modern IoT architectures require security to be embedded across all layers, from device identity and boot processes through to Cloud authentication and data access controls.
This includes secure provisioning, trusted device identity, encrypted communication, and controlled update mechanisms.
When security is designed as part of the system architecture, rather than applied after deployment, it becomes more consistent and scalable. When it is added later, it often results in fragmented implementations that are difficult to manage across large device fleets.
Designing IoT systems as complete architectures
Many of the challenges seen in IoT deployments are not caused by insufficient technology, but by systems that are not designed holistically from the outset.
When hardware, software, connectivity, data flow, security, and lifecycle management are treated as separate concerns, systems tend to become increasingly complex over time. Issues that are not visible in early-stage deployments often emerge at scale, where integration and coordination become critical.
A more effective approach is to design IoT systems as complete architectures, with explicit consideration of how all components interact across edge, cloud, and lifecycle domains.
This requires early alignment on system behaviour, data flow, and operational constraints, rather than treating these as implementation details.
As IoT systems continue to scale in complexity, the ability to define coherent architectures at the outset becomes increasingly important. When this is achieved, systems are more predictable in operation, easier to maintain, and better able to evolve over time without requiring continuous redesign.
Author biography:
Martin Grossen is Director, Embedded Software and Cloud at Avnet Silica
