Pilgrim
12th July 2018

The past

Historically, companies deploying connected devices had to build their own operational tools because there were no off-the-shelf solutions. They often didn’t explicitly decide to do this, but a developer script for managing 10 devices somehow became a set of scripts for managing 100 devices which then became a simple web app for managing 1000 devices, and so on.

The total cost in time and money was huge – let alone the “opportunity cost” of diverting precious developers away from their core mission of building the company’s unique value. And worst of all, these tools inevitably proved inadequate as the company grew:

  • Disenfranchising all the business-people who needed daily answers to questions about the device estate – requiring them to beg developers to change scripts etc.
  • Encumbering the company with major ongoing development costs

The Web has spawned many off-the-shelf Business Intelligence (BI) analytics tools – available open-source and/or as-a-service – so it’s natural to wonder if perhaps these can be pressed into service? The analysis below shows that three of the most popular BI tools have significant limitations when it comes to managing IoT.

Make no mistake, these tools are great pieces of infrastructure (IaaS) for general analytics, but they are not by any stretch complete applications built to manage IoT (SaaS). Their principle limitation is that they require upfront developer support and ongoing “developer in the loop” support whenever business-people have new questions, wiring-in ongoing business friction and cost.

1. Splunk

Splunk is a popular big data analytics platform aimed at Data Scientists which charges per GB ingested per month. Its philosophy is to always process from the raw data. It was specifically designed to deal with streaming data such as that from server log files – not IoT data specifically. It very much needs a developer to set it up and to manage the pipeline of data on an ongoing basis.

Splunk as a service is expensive on ingestion because it doesn’t charge for querying. It is always possible to reduce ingestion bandwidth by running code in lambdas to pre-calculate values as data is ingested, but that would incur more developer cost, and exacerbate the developer-in-the-loop problem.

2. Quicksight + AWS IoT Analytics

Here we assume that you hire a developer to build an AWS IoT Analytics pipeline to preprocess your data, and then view it using Quicksight, a generic BI tool which is great for making high-level dashboards. a non-developer can then ask basic questions of the data (timespan, filter, basic operations).

Limitations:

  • All data has to be pre-structured to suit the queries. So:
    • Likely need ongoing developer time
    • When new queries are needed, historical data will not be restructured, so cannot easily ask new questions about existing data
    • Can’t do “duration” operations, e.g. measure number of devices offline, or how long a device has been offline.
  • Quicksight doesn’t treat time as a first-class citizen, so e.g. if you plot against time then you get “per-day” resolution. Can’t define “time bins” on the fly e.g. compare devices every 5 minutes (any binning has to be done on the way in).
  • Quicksight only imports once an hour, so data will be delayed by up to that amount.

…plus Lambdas and DynamoDB

Most of the above limitations (e.g. measuring uptime) could be overcome by running an AWS Lambda every time new data arrives, comparing current state with historical state stored in DynamoDB. But this has very significant initial and ongoing developer costs, plus ongoing cost for running the lambda and accessing DynamoDB.

3. InfluxDB + Grafana

InfluxDB is a well-respected time-series database aimed at Developers, available as-a-Service, intended to visualise telemetry. New queries have to be implemented by developers, then a business person familiar with SQL can use Grafana to query. There would be very significant developer cost to “bake” Influx into a platform, and e.g. to pre-process operational data such as uptime into the Kapacitor pipeline. A key limitation is that one must decide up-front whether each incoming property is metadata or telemetry (can’t “group by” telemetry).

Comparison

These numbers are for 10,000 devices, from this spreadsheet.

 

Splunk
AWS Quicksight +
IoT Analytics
InfluxDB +
Grafana
DevicePilot
Setup cost
$$$
$$
$$
$0
Running cost of OM suite (per device per month)
$2.71
$1.66
$1.51
< $1.00
Easy for normal business people to use
×
×
×
Able to measure key IoT metrics such as “up-time”
×
×
×
Ask new types of question without developer input
×
×
×
Ask new questions about existing data
×
×
×
Lifecycle Automation
×
×

 

Conclusion

This analysis demonstrates why the best solution for managing IoT devices is a SaaS tool specifically designed and optimised for that purpose, rather than trying to build a custom application from generic IaaS database technology. Companies like yours are already using DevicePilot to increase customer-satisfaction and reduce costs – so start a free trial today at devicepilot.com.