IoT industry trends in performance testing

By Pilgrim - March 25, 2022

Our CEO Pilgrim was interviewed by Josh Taubenheim of MachNation about how IoT vendors can scale while ensuring high reliability. Topics they discussed include:

  • Challenges of scaling IoT - from Building to Running
  • Does it work (and if not, why not?). Lots of ways things can go wrong.
  • DevicePilot - a SaaS tool for service monitoring and management
  • Why it's important - devices are there to deliver a service
  • IoT these days is less about whizzy bells & whistles, more about the life-cycle of a device
  • Do IoT tools and organisational requirements change with scale?
  • "Manual" approach has merits initially, but wheels will fall off as you scale if you stay manual
  • Testing IoT frameworks at scale
  • Ongoing monitoring essential
  • KPIs - internal, informal metrics over time for what is good-enough - often start quite technical e.g. "uptime". 
  • At greater scale, consolidated channel-partner customers will enforce harder SLAs. Need to manage to them.
  • Quality does cost money, are you aiming for "two nines"?
  • Need to move beyond purely technical metrics to business metrics - e.g. from uptime to site availability
  • Tech metrics are "necessary but not sufficient"
  • Performance testing - continuous monitoring or "point a million devices at an endpoint and see what happens"
  • Simulated device estates for testing
  • Have to plan ahead - can't have confidence your architecture will scale until you test it (with synthetic devices). Maybe 10x today's load.
  • Future of performance testing, how will tools evolve? 
  • IoT has so many moving parts, so many industries & vendors. Compared with telco, servers, web it's been slow, because they were so much more homogenous. 
  • Starting to see standard parts of the stack, what are they called, how do they fit together, who makes them.
  • Establishing category names makes it easier for customers to find vendors and vice versa  

Listen to Pilgrim (Episode 6) alongside other interviewees, from companies including PTC, Waylay, Losant and Software AG in the MachNation Tempest podcast library


See how DevicePilot can make the difference


Industry leaders trust DevicePilot to help them improve the quality of the service they deliver at scale.

  • Eliminate revenue loss
  • Deliver a better service with the same human resource
  • Focus on growth and not firefighting
  • Get customer satisfaction through the roof

Book your personalised demo now and discover how DevicePilot can help you scale your connected business

Erik in a circle-1

Erik Fairbairn, CEO at POD Point:
Achieved 99% uptime across device estate

"We're totally data driven at POD Point, and if we can answer a question using data then we think that’s the best way - there’s no guesswork and you can use the facts.

Our DevicePilot dashboards have really let us get that actionable insight out of our devices."