Application Architecture
Contents
The Simple IoT Go application is a single binary with embedded assets. The database and NATS server are also embedded by default for easy deployment. There are five main parts to a Simple IoT application:
- NATS Message Bus: all data goes through this making it very easy to observe the system.
- Store: persists the data for the system, merges incoming data, maintains node hash values for synchronization, rules engine, etc. (the rules engine may eventually move to a client)
- Clients: interact with other devices/systems such as Modbus, 1-wire, etc. This is where most of the functionality in a SIOT system lives, and where you add your custom functionality. Clients can exist inside the Simple IoT application or as external processes written in any language that connect via NATS. Clients are represented by a node (and optionally child nodes) in the SIOT store. When a node is updated, its respective clients are updated with the new information. Likewise, when a client has new information, it sends that out to be stored and used by other nodes/instances as needed.
- HTTP API: provides a way for HTTP clients to interact with the system.
- Web UI: Provides a user interface for users to interact with the system. Currently it uses the HTTP API, but will eventually connect directly to NATS.
The simplicity of this architecture makes it easy to extend with new functionality by writing a new client. Following the constraints of storing data as nodes and points ensures all data is visible and readable by other clients, as well as being automatically synchronized to upstream instances.
Application Lifecycle
Simple IoT uses the
Run()/Stop()
pattern for any long running processes. With any long running process, it is
important to not only Start it, but also to be able to cleanly Stop it. This is
important for testing, but is also good practice. Nothing runs forever so we
should never operate under this illusion. The
oklog/run packaged is used to start and shutdown
these processes concurrently. Dependencies between processes should be minimized
where possible through retries. If there are hard dependencies, these can be
managed with WaitStart()/WaitStop() functions. See
server.go
for an example.
NATS lends itself very well to a decoupled application architecture because the NATS clients will buffer messages for some time until the server is available. Thus we can start all the processes that use a NATS client without waiting for the server to be available first.
Long term, a NATS API that indicates the status of various parts (rules engine, etc.) of the system would be beneficial. If there are dependencies between processes, this can be managed inside the process instead of in the code that starts/stops the processes.
NATS Integration
The NATS API details the NATS subjects used by the system.
Echo concerns
Any time you potentially have two sources modifying the same resource (a node), you need to be concerned with echo'd messages. This is a common occurance in Simple IoT. Because another resource may modify a node, typically a client needs to subscribe to the node messages as well. This means when it sends a message, it will typically be echo'd back. See the client documentation for ideas on how to handle the echo problem.
The server.NewServer function returns a nats connection. This connection is used throughout the application and does not have the NoEcho option set.
User Interface
Currently, the User Interface is implemented using a Single Page Architecture (SPA) Web Application. This keeps the backend and frontend implementations mostly independent. See User Interface and Frontend for more information.
There are many web architectures to chose from and web technology is advancing at a rapid pace. SPAs are not in vogue right now and more complex architectures are promoted such as Next.js, SveltKit, Deno Fresh, etc. Concerns with SPAs include large initial load and stability (if frontend code crashes, everything quits working). These concerns are valid if using Javascript, but with Elm these concerns are minimal as Elm compiles to very small bundles, and run time exceptions are extremely rare. This allows us to use a simple web architecture with minimal coupling to the backend and minimal build complexity. And it will be a long time until we write enough Elm code that bundle size matters.
A decoupled SPA UI architecture is also very natural in Simple IoT as IoT systems are inherently distributed. The frontend is just another client, much the same as a separate machine learning process, a downstream instance, a scripting process, etc.