The Internet of Things has driven many quality assurance teams to reevaluate their current test management strategies. The multitude of platforms and devices in play within an IoT environment make traditional approaches to testing untenable, necessitating significant changes. Perhaps no single QA component will be as critical to effective IoT testing as the use of automation.
Test automation has become a fixture of modern quality assurance, in large part because IT organizations now have to churn out applications as well as updates and patches at a torrid pace. This shift to fast-paced development, testing and deployment is one big result of rising adoption of agile methodologies, designed to boost competitiveness.
A 2015 online survey of 601 IT professionals, conducted by HP, found that 51 percent of respondents were "leaning toward agile," while 16 percent reported adhering to "pure agile" and another 24 percent were pursuing a hybrid (agile and waterfall) approach. Agile management, rather than waterfall, is emerging as the norm.
The most cited reasons for taking up agile included the desire to increase collaboration between siloed departments, improve customer satisfaction and software quality and shorten overall time to market. Indeed, automation provides simplification and scalability to offset the new features added in each agile sprint.
Test automation: A natural fit for the Internet of Things
The ongoing trend toward test automation as one key cog within a broader culture of agile devops will likely only continue, especially as the Internet of Things comes into focus. There are several big reasons to expect test automation to be at the heart of IoT development:
The IoT brings a uniquely complex set of testing requirements to the table. More specifically, it combines the characteristics of latter-day Web and IT services (e.g., cloud-based applications) with those of traditional embedded systems. As such, a diverse set of testing regimens will be needed for adequate coverage.
The size of the IoT is unprecedented. IT research firm Gartner has estimated that there could be more than 26 billion devices connected to the IoT by 2020. This immense scale makes automation a no-brainer for running tests on everything from home thermostats to the sensors that might be placed in a refrigerated truck.
Connectivity and security require close attention in the IoT. A wide range of protocols including Wi-Fi, Bluetooth, 4G LTE, ZigBee, etc. are in play for the billions of new devices coming online. Testers must account for a variety of connectivity scenarios while also focusing on the vast attack surfaces available to cyber criminals.
Overall, it makes sense to think of the IoT as not only a huge set of devices and services interacting with one another, but one that is multilayered and in need of automated testing for the components of each layer. The "things" in the IoT - e.g., the networked devices and sensors - get most of the attention, yet they are basically like the middle part of a giant sandwich. "Below" is the computing infrastructure that contains applications and analytics, while "above" are the local and global communications apparatuses that connect devices to each other and to networks like the Internet or a corporate WAN.
IoT testing requires proper oversight of each of these layers, with sets of tests that can automatically model and vet a huge range of possible scenarios. For example, a tester may wish to simulate an IoT system that features a network connection to a gateway node and a wireless mesh. Many virtual networks may operate in tandem with a physical one to rapidly simulate hundreds or even thousands of IoT nodes. From there, testers can pay particular attention to matters such as emulating the long idle times of IoT sensors and seeing how to optimize their accompanying applications and services for metrics such as energy efficiency, connection strength and speed.
Not just software: Why IoT must be both automated and holistic
The stakes for effective test automation are high with the IoT because organizations are not only performing software testing but also vetting an entire ecosystem of connections, devices and real-time scenarios. QA teams have already gotten a feel for this new normal in working with mobile devices, which by their nature encounter dynamic conditions such as fluctuating cellular connectivity. The IoT ups the ante by introducing myriad other sensors and services into the picture.
Accordingly, tests can not stop at simply validating the functionality of software within tightly controlled settings. QA teams could have a good understanding of the hardware and protocols in play but still need additional time and experience to understand how all of these moving parts behave in the volatile environment of the IoT. Lessons learned from working big data services and analytics could be useful here. That is, in addition to seeing if a system basically works, testers must also check to see if it is scalable and reliable under common as well as unusual circumstances.
Enterprise test management software is a good starting point for IoT testing. Combining these tools with agile processes and a DevOps culture can help IT organizations map out a feasible test automation strategy for the IoT.