May the Force-directed Graph Be with You!–Visualizing IoT Device Relationships
Most of us intuitively visualize people and social networks with a relationship graph, yet few of us extend the analogy to the IoT space. There is a mass of graphs around “things.” Uber and Lyft are typical examples of Internet of Things that connect you in real-time to a graph of geo-located cars in your vicinity.
It’s all centered around relationships. The ability to visualize these very complex and dynamic systems is key to securing, maintaining, and deriving maximum insight from them. Engineers, data scientists and business analysts must react in real-time to IoT events–they don’t have the luxury of wading through the process of creating a diagram. To get a true handle on IoT events, graph-based visualization must be democratized. Additionally, creating an interactive, easy-to-understand representation of real-time, historical, or any combination of data must be as simple as generating an Excel spreadsheet.
Because IoT-related decisions are often highly complex and time-sensitive, data visualization cannot be approached as something that you build. For that matter, it can’t be viewed as static. We’re talking about systems that update in real-time, sometimes to massive effect with petabytes of data. Nor can data sets be viewed in isolation, as IoT data is always part of a much wider system. To derive actionable insight, different data sets must frequently be compared–both those generated in real-time and those that may have been generated years prior. Business analysts, data scientists and engineers must be able to build powerful, interactive, instantly refreshable visualizations with no more than a few mouse clicks.
Visualizing the answer to the “ultimate question”
IoT systems can involve hundreds of trillions of data points, so seeking one single way to visualize IoT data is kind of like expecting a Hitchhiker’s Guide to the Galaxy-styled answer to the ultimate question. Understanding these incredibly complicated systems requires a number of techniques, so let’s talk about a few particularly important ones.
Let it flow
A Flow diagram represents flows within a system of an item such as data, energy, vehicles or people. While flow diagrams have been used for years to illustrate interplay within complex systems such as engines, IoT may be among the most dynamic applications in which this visualization technique can be applied. A Sankey diagram is a flow diagram that illustrates the proportionality of flow quantity by the size of arrows within the graph:
Named after Matthew Henry Phineas Riall Sankey, who used it to examine a steam engine’s workings, it might be used to visualize various flows within an IoT system, such as:
- The volumes of data being pipelined from various nodes
- Energy consumption based on data streaming from sensors on factory machines
- Electricity flowing in from panels on a solar grid
- Movement of vehicles in a fleet through transportation routes
- Flow of goods among factories, warehouses and stores
Think about smart buildings sensors. If one sensor reports that an room is getting warmer, would you request to push the cooling system or decrease the thermostat? Probably what flows through your mind is:
- How many people are in the room?
- Is the cooling system not coming on?
You will be surprised to know that IoT events that are flowing continuously into your systems are not very well represented with traditional two dimensions charts but requires flows visualization such as Sankey diagrams.
Voronoi diagrams are named after Russian mathematician Georgy Fedosievych Voronyi and described by Wikipedia as “a partitioning of a plane into regions based on distance to points in a specific subset of the plane…for each [point] there is a corresponding region consisting of all points closer to that seed than to any other.” If that makes your head spin, think of a Voronoi diagram as a map composed of “cells” resembling a honeycomb. Within each cell is a point called a “site,” and if you’re inside that cell you know that the closest site to you is the one that lies within the cell that you’re in:
Without the cells for reference, it would be difficult to look at something like this and tell which of the points (or sites) you were closest to:
This could be problematic if, say, you were injured and each site on the diagram represented a hospital. But with the Voronoi cells, no matter where on the diagram you land, you immediately know the closest point to where you are. While it’s easier to grasp this concept initially if we think about the diagram as a geospatial map, the sites can represent just about any value or data type, and might not actually be a single value, but rather a vertex or mean of a group of clustered values. In IoT systems, Voronoi diagrams might be used to:
- Identify regional energy usage patterns
- Derive the capacity of a wireless network
- Aid in autonomous navigation of vehicles and drones
Let’s take the example of proactive fleet maintenance. You have thousands of heavy equipments from trucks to tractors to cranes…at work around the world every minutes year round. You need to know a set of 100 parameters (temperatures, RPM, oil level, weather,…) in real-time to predict the health of each equipment, perform proactive maintenance and avoid costly recover and repair. One “many-to-one” monitoring practice is to perform “peer health monitoring” where you will cluster devices of similar type and usage together and detect outliers on a min of 100 parameters. This is next to impossible to represent with traditional charts, with a voronoi chart you can easily distinguish any anomaly even if you have billions of devices.
May the Force-directed graph be with you!
Force-directed graphs are perhaps the coolest, yet toughest to conceptualize. Put simply, they allow us to visualize dynamic relationships of any type within a system, from those that exist between university alumni to the forces that attract and repel subatomic particles. They’re kind of like virtual lattices that can be poked, prodded or compressed, yet consistently “bounce back” to a state of equilibrium.
Imagine that the energy you put into interpersonal relationships was represented as a force-directed graph, with lines between you and the various people in your life. Things are going along normally, then you have the work-week from hell which doubles the amount of energy you spend on work relationships. That energy comes at the expense of friends, family, etc. So in a graph, every time you stretched the “work-relationship lines” it would tug on every other relationship line on the graph, creating a new shape for the entire graph. Eventually, things get back to normal, and the graph of your life returns to equilibrium.
As IoT systems are typically composed of elements that are interrelated, the uses for force-directed visualization of data abound. A “Smart City” for instance might be using IoT data to improve traffic flow. It wouldn’t be enough to analyze data on individual roads–to get a true picture, you’d need to understand how heavy or light traffic at a single intersection influenced traffic in the entire city. With a force-directed graph, you could represent how traffic at every single intersection in the city affected another, and devise a model to improve traffic flow.
Logtrust has democratized graph-based visualization
Logtrust not only makes it possible to collect massive amounts of data from IoT systems, but also to visualize, understand and take action on it. Logtrust has democratized graph-based data visualization, enabling organizations with IoT systems to very quickly enact a myriad of visualizations so that they can act understand and act on events as they are processed in an event lake. This combination of processing, storing and analyzing makes it possible to address any concern from security to compliance, monitoring and business intelligence in a single platform. Logtrust provides the most advanced visualization without the need for technical or programming knowledge:
- Drag and drop UI and intuitive visualization to detect deviations and spot hidden data relationships on high volume, high velocity data streams
- Interactive dashboards can be shared to increase shared intelligence and collaboration
- A single view to visualize, correlate, analyze and report on all your data in real time, regardless of format, across business units
Case in Point:
Logtrust is working with a Fortune 500 Global Telco provider with more than 4.5 million digital devices in the field, delivering billions of data streams for TV and Video On-Demand services. Logtrust created an event lake data layer that enables it to operate an IoT system to manage more that 20 million customers’ data at scale, and improve Quality of Experience (QoE) and Quality of Service (QoS) – in real-time. Logtrust not only enables the company to process massive data sets in real-time, but it also allows analysts and customer service reps to slice and dice various data sets using a myriad of visualization techniques, all of which can be generated immediately:
- To visualize the flow of data among connectivity routes throughout different regions, neighborhoods and boxes, they can instantly generate a Sankey diagram.
- To create a map that identifies different regions by TV watching hours or preferences (so that they can tailor advertising in those regions), they can quickly spin up a Voronoi plot.
- To conceptualize more complex and dynamic relationships between customers, sales reps, service technicians, customer service reps, or any other combination of elements, they can rapidly produce a force-directed graph for unique and valuable insight.
Using Logtrust, the Telco giant analyzes in real-time billions of data for millions of customers each day, and has reduced call volume, improved QoE/S, and saved millions of dollars a year.
Any way you slice the data, a picture is worth a thousand words–the more “pictures” you can generate, the more value you can derive from your IoT system.