The usage of products, solutions and systems based on Internet of Things (IoT) technologies is growing rapidly, and providing breakthrough value across domains such as smart city management, manufacturing, agriculture, retail and transportation. When it comes to software testing, this IoT era brings in a whole new set of challenges and needs a fresh approach so as to ensure quality-assured systems.
Security lapses in IoT systems could have very serious repercussions, and yet, many IoT devices are currently vulnerable to attacks. A recent report by Armis estimates that half a billion IoT devices inside enterprises could be vulnerable, and may allow hackers to bypass the network firewall. The functioning of IoT systems is also critical, as a malfunctioning one could put human lives at risk, for example – a traffic or hospital management system.
IoT systems comprise of communication, applications, and data analysis for monitoring and managing a variety of ‘things’. So IoT testing protocols must also cover all these various aspects.
It is extremely important to have QA testing practices that are designed and planned in alignment with product design. These can play a valuable role in mitigating risk, ensuring a properly functioning system and achievement of the defined goals.
The following factors make testing of IoT systems both critical as well as challenging
- Diverse devices
- Testing for wireless connectivity scenarios
- Real time environments
An IoT network could include many different kinds of devices, including home appliances, temperature sensors, wearable devices, to name just a few. How the software will function may need to be tested on each of these devices. It is also necessary to test the integrations and interfaces to ensure universal functionality across all touch points. Connected devices also have to be tested for acceleration, vibration, stability, repetition, stress, memory and endurance factors.
Connectivity in an IoT system is dependent on many different wireless standards. For example, an IoT-enabled device might have to interface with Wi-Fi, Bluetooth, ZigBee and/or 4G LTE. So software testing has to consider many different scenarios related to the wireless connectivity. Some questions that need consideration are: what happens to data when a connection is unexpectedly dropped? Is it saved and properly stored? Where will it pick up once service is restored?
IoT devices could be operating in challenging weather or physical conditions, such as a high temperature environment in a factory, or dusty surroundings in open areas. QA teams must test the full range of environmental vulnerabilities, and estimate whether exposure will affect functionality.
Test management for IoT systems
- Preventing vulnerabilities
- Testing on different devices
- Big Data testing
- Real world testing
Security of the network and data is critical in IoT systems. QA teams must estimate potential vulnerabilities and guard against them. Some of the measures may include:
– Encryption between devices, or between device and server
– Closing ports to internet communications when not in use
– Enforcing strict password rules
As there are many different devices, the software may need to be tested on each of them, as standardization is not in place. The device category where a degree of standardization exists is the smartphone. Software can be tested by executing code on a phone or tablet directly or with a software based emulator. Hardware as well as emulator testing may be used for UI layout, data exchange and geolocation interaction.
Testing may involve the granular testing of chips and sensors or the test coverage for multi-platform data flows. Calibrated stimuli are used to observe and measure parameters such as electrical signal shape and strength, memory length, and display accuracy and appearance.
IoT systems generate a huge volume of data. This data could be of many different types and this has to be considered while testing. Testing of each type of data set requires the right techniques to check the data ingestion, data migration, data integration, data standardization, data homogenization etc.
The test plans now need to incorporate testing in real conditions, for example, movement between different wireless networks, or testing of home automation systems in the actual home layout. Service visualization tools can be used to model different types of physical layouts, sensors and device states.
The Future of IoT Testing
As IoT generates a whole new set of testing requirements, automated testing has an important role to play. Simulation models will also be very important, given the challenges with real-time testing. Testing tools and procedures will need to verify various communication protocols, including WiFi, Bluetooth, CDMA, and 3G.
IoT testing will throw up a number of new challenges, but also opportunities for better testing practices. Those products or systems that follow sound QA practices will create more reliable systems as well as business gains for themselves.