Navigating mobile software testing
Many companies have the misperception that an app is part of any attempt to enter the mobile space.
By Mario Matthee, Chief Operating Officer of the DVT Global Testing Centre.
The mobile software testing market is experiencing somewhat of a Jekyll and Hyde complex at the moment.
On the one hand, mobile device growth is continuing apace, with predictions that mobile users will overtake PC and laptop users for most business and web-related tasks having long since been realised. On the other, the app boom seems to be well and truly over, with recent figures in the US showing users are downloading an average of zero apps per month.
This isn’t difficult to explain, it simply means the mobile market, which began its upward trajectory more than eight years ago, has reached saturation, and most users are now familiar with the apps they need and use every day. There are some exceptions – like Snapchat and Uber – that defy the trend and are still growing at a phenomenal rate, but unless you’re very good or very lucky, it’s going to be difficult to get your new app noticed and downloaded among the crowd.
How does this affect mobile testing, you ask? In two fundamental ways. First, the drop-off in new app development means companies have a decision to make when it comes to reaching out to their customers through mobile platforms. Apps are no longer the first step in creating a mobile presence; for many companies a responsive mobi (mobile-oriented) site makes more sense.
Secondly, device selection is vital, and increasingly so. The rapid growth and maturity of mobile devices in general and smartphones, in particular, has seen the market ultimately settle on two major platforms – iOS and Android. Some smaller platforms such as Windows Mobile and Blackberry are shrinking and even (in the case of Blackberry) migrating their users to various flavours of Android.
Because of this polarity, and the loyalty of most users to one platform or another, developing and testing native apps is not always the smart choice, especially for newcomers to the mobile space. But if testing on a limited number of devices is counter-intuitive, testing on a very large number of devices is often prohibitively expensive.
And so the starting point for any conversation on mobility and mobile testing should always be a company’s digital strategy. Unfortunately, most companies I speak with today don’t have a fully formed digital strategy, and those that do are half-cooked or based on the perception that an app is part and parcel of any attempt to enter the mobile space. The truth is that a well-built, responsive and intuitive mobile website is almost as important – if not more so – and can also perform most if not all the functions of an app, depending on the type of business it’s used for.
Deciding between apps, websites, or a combination of the two is just one of the challenges. A comprehensive digital strategy also needs to cover factors like device management, device types, usability testing and automation.
From a testing perspective, device management is critical because mobile devices are susceptible to damage, loss and theft more frequently than almost any other device type. It may seem inconsequential, but given the high cost of devices and the risk of valuable IP taking a walk at a critical development stage is very real.
The issue of device types is important regardless of the software you’re testing, be it an app, a mobile site or a desktop site on mobile devices. Even a closed platform like iOS comes with the challenge of users with previous generations of iPhone and iPad, and multiple iterations of previous generations as well.
A modern, responsive mobi site or app might light up the screen of the latest iPhone, but could bring previous legacy iPhones with older versions of iOS to a standstill. And iOS is fairly straightforward compared to the permutations of the hundreds or thousands of Android devices from dozens of different manufacturers on the market today.
Once device management and types have been narrowed down to a manageable grouping, usability testing on these devices is a third major consideration. This is where mobility testing also differs the most from other forms of software testing, because it’s necessarily hands-on. Yes, developers can test their mobile apps or websites on simulators and the odd device, but neither of these options are anywhere close to sufficient for a proper functional test of a new (or new version) of the software.
It’s almost impossible to remotely test mobile software, not because the technology is lacking, but because usability is such a big factor in the success or otherwise of a mobile app or website. And when it comes to usability, that means testing by experienced human operators, not machines.
Which brings me to the last point, automation. Even if it were practical to automate some parts of the mobile testing process, the rapid rate of change in both devices and apps (and websites) means by the time you’ve invested in solid testing scripts for your software, a new version rolls out, your users have upgraded to new devices, and a new OS has been released. That’s not to say automation won’t play an important role in your mobility strategy, but you’ll probably find manual testing plays a much bigger one.
By now you’re probably getting the sense that jumping into the mobile space – or growing your current mobile presence – is a much bigger ask than you thought, and you’d be right. Navigating the mobile testing minefield can be a nightmare if you don’t have a solid, thought-out digital strategy that informs every decision you make based on the value of the investment to the business.
A good place to start would be finding a likeminded partner with the experience to guide you through the creation or refinement of your digital strategy, before giving you access to the resources you’ll need to make it the success you need it to be.
This article was published exclusively for ITWeb on 13 September 2016.