Take this app and... Jul 8, 2016
5 reasons why users are ready totrash your app
Every morning, I launch my NPR news app, press play, and—nothing happens. Each
time, I must go to another screen to re-select my local NPR news station, return to
the main screen, and press play again. Only then does the broadcast start. I
shouldn’t have to do that. The app is supposed to remember my local station.
Nevertheless, the workaround has become part of my morning routine because I
am willing to put up with it to hear the news.
But when an ABC News app began interrupting me with notices that it had stopped
functioning—even when I wasn’t running it—I swiped it into the trash without a
second thought. My life is full of software issues that somehow must have survived
the quality assurance gauntlet, and the incidence of such annoyances only seems to
be increasing.
2016 World Quality Report -latest trends in QA & testing
What about the apps you or your team have built and deployed? Are they simply
being tolerated by irritated users? How many times did your app get swiped into the
trash, without you even knowing it? And what should you do about it?
The software quality conundrum
Ask a typical consumer whether the state of software quality has improved and you
may not like what you hear. Over the last six months, I’ve personally had to deal
with a barrage of software-related issues, and I’m not talking about apps from mom-
and-pop development shops. Some problems I've experienced come from apps built
by some of Silicon Valley’s biggest software companies —the ones held up as the
true innovators in software development.
In my day job, I speak with software quality experts, attend conferences, and talk to
developers and software quality experts in leading-edge businesses who sing the
praises of using DevOps, agile, and continuous delivery pipelines to get better-
quality software to market faster. But at home, someone is always struggling with
one application glitch or another. And it’s not just mobile and desktop apps. The
Internet of Things has introduced its own series of problems, from my home
security cameras that mistake noise for motion and issue false alerts to the smart
thermostat that just can’t seem to maintain its connection to my router.
“It’s not just you,” said Michael Cooper, chief quality officer at Healthcare IT
Leaders and Run Consultants, after hearing my long litany of software issues. “The
issues you describe seem to have happened to a lot of people. It’s a real problem.”
Cooper, who has led QA teams at HP, T-Mobile USA, and FICO, feels the burn. He
had to deal with a programmable thermostat that shut off his home heat last winter
due to a software glitch and a Windows 10 upgrade that corrupted all of the data on
his hard drive.
I asked him to weigh in on five of the most frustrating software-related issues I have
faced recently. I wanted to know what went wrong, why these issues keep cropping
up with such alarming frequency, and what software developers can do to prevent
similar issues in their own applications.
1. The mobile OS upgrade from hell
Not long after I installed Google’s Android Lollipop update, things began to go
horribly wrong. At first, just a few apps began to respond slowly or crash. But the
situation gradually worsened over the next few days, until my phone became
practically unusable. Wiping it and reinstalling the software helped initially, but
then the glitches would just begin anew.
At one point, the Android phone dialer app froze in a Windows-like blue screen of
death; at other times the screen went black after I initiated a call, making it
impossible to hang up. The last straw was when my phone crashed in the middle of
an important conference call. I powered it back up, only to see this dialog:
“Optimizing app 1 of 135…” More than a half-hour later, the phone finally came back
online. But by then I had missed a call with my boss, and I lost even more time after
deciding to wipe and rebuild the phone—again.
For business users such as me, this is the downside of IT consumerization. The new
features in Lollipop, while nice to have, did not deliver any quantifiable business
value. And they introduced risks. What I value most is stability and reliability, not
cool new features. I suspect I'm not alone in this preference.
For this reason, I initially resisted the nagging OS upgrade messages. But the carrier
said there was no way to turn those reminders off, so I finally gave in. Only after the
problems began did I realize that there was no going back, no “restore to last known
good state.”
My carrier blamed the mobile operating system and the phone manufacturer for my
woes. Its solution: Buy a new phone, just 14 months into a 24-month contract for a
piece of business equipment that cost $700. Ouch. After several calls to tech
support and three visits to the carrier’s store, I ended up dumping the phone.
Here’s Cooper’s take:
What went wrong: Most likely there were some incompatibilities with the
Android code and the Qualcomm chipset in my device. Clearing the cache
might have resolved the issues. Unfortunately, that advice came too late for
me.
Why it happened: Google, device manufacturers, and wireless carriers don’t
test Android upgrades on every device that’s out there. Older devices are
rarely tested for compatibility with new versions of Android, and most
device makers add proprietary code to Android, increasing the potential for
upgrade-related issues. Even with new devices, Cooper says, “There is often
a rush to market, and testing is limited.”
How software development teams can prevent it: For Android’s software
developers, it’s important to understand which devices are most popular and
test the top 20 percent. That should cover more than 80 percent of your
users. The best ways to get this level of test coverage are through
crowdsourced testing, mobile test automation, and the use of a cloud-based
mobile testing lab.
Device makers should collaborate with OS makers to test their new device lineup
on the latest and upcoming OS versions, Cooper added. Carriers and device makers
can reduce problems by limiting customization of the Android operating system.
Apple doesn’t allow nonstandard versions of IoS, which is why such issues occur less
often with iPhones, he said.
As for users, they should wait to upgrade until bugs have been worked out for their
phone, Cooper advised, and wipe the cache immediately after an upgrade.
2. The online banking upgrade that threw easeof use under the bus
I loved my bank’s old bill-paying system for its simplicity. Once I had set up my
payees, I brought up the list, typed in the amount to pay and the payment date in
two little boxes next to each vendor’s name, clicked the “pay” button at the bottom
of the window, and I was done.
With the new bill-pay system, I must click on each biller name individually to open a
pop-up window, then click the buttons on a calculator image to enter the dollar
amount, press the “enter” button, and then open another window to set the
payment date before clicking “submit.” To pay the next biller, I must return to the
main bill-pay screen and start the process all over again.
Here's what Cooper had to say:
What went wrong: Most likely this is due to poor design, with no
consideration of its impact on the user.
Why it happened: It could be due to a lack of user experience (UX) training
and, most likely, the business performed no UX testing.
How software development teams can prevent it: Hire professional UX
designers and perform UX testing with feedback. The most customer-centric
companies, such as Intuit and Apple, are known for UX testing and use of
focus groups.
3. E-billing Catch-22: After choosing e-billingover paper, I get neither
When I signed up to receive e-bills from one payee in my bank’s online bill pay
system, the paper bills stopped arriving, but the e-bills didn’t show up in the system.
When I attempted to turn off the e-billing option, the software complained: “You
must receive at least one online bill before e-billing can be turned off.” I then
attempted to delete the biller, only to be blocked again. The system’s response:
“Biller can’t be removed until online e-billing is turned off.”
Cooper had some ideas:
What went wrong: It’s possible that my spam filter blocked my first e-bill, or
corrupted data might have prevented the system from sending the e-bill to
the correct email address.
Why it happened: It is hard to anticipate all of the issues that can go wrong
with e-billing. There are usually several systems involved, and all must be
integration-tested from end to end, including the actual sending of the bills
and the process of switching from e-billing to paper billing and back. Often in
a testing cycle, teams don’t budget enough time to run several billing cycles,
including the mailing of real paper bills and the e-mailing and verification of
e-bills.
How software development teams can prevent it: End-to-end testing must
include switching between e-billing and paper billing options and back.
Amdocs, which tested billing for T-Mobile, created a logical date in its system
so it could test multiple billing cycles with representative synthetic data in
order to avoid exactly this type of situation, Cooper said.
4. Gaming software rejects payment, dumpsuser data
When my nephew received a popular PC game for Christmas, my brother-in-law
went online to register and pay for it. He filled out the profile information for
himself and his young son, entered his credit card information, and promptly
received a “payment rejected” error. When he returned to the registration screen to
try again, all of his personal profile and billing data had been discarded. He repeated
the process using a different credit card, with the same result. As it turned out, both
credit cards were fine. But this particular user ended up re-entering his profile data
three times before the issue was finally resolved, and then only after making a call
to technical support.
Cooper suggested the following reasons for my brother-in-law's hassle:
What went wrong: To process a payment, the vendor must make a call to a
credit card processor. The credit card transaction must pass several fraud
checks, and all of the information needs to match before it can be processed.
It must be clear to the user that, for example, the address entered must
exactly match the billing address on file with the credit card company. A
temporary outage in the connection to the payment system also could have
led to the problem. While the credit card information must disappear for
security reasons if the request is rejected, other data that the user has
entered could have been retained.
Why it happened: Many companies stop testing before making an actual
payment because they don’t want to spend the money during the test,
Cooper said.
How software development teams can prevent it: Get test accounts from
the credit card processing company so you’re not actually getting charged
during testing. Also, be careful not to trigger fraud protection during credit
card testing. End-to-end testing is key, and tests need to be run against all
major payment types you plan to accept.
5. Software upgrade corrupts user data
My migration from Windows to the Mac version of a popular personal finance
program turned into a data conversion debacle. The Mac version could not read the
data file created in the Windows format without a conversion step. When that
process failed, the vendor offered to do it for me, and I gladly uploaded my file. But
when the Mac-ready file came back, "memorized transactions"—a feature that
recognizes a transaction and automatically assigns it to the proper reporting
category—didn’t remember what went where. The categories came over, but the
logic on when to apply them didn’t. Tech support was unable to help, and I resigned
myself to manually recreating all of my memorized transactions as I created new
entries. But when I realized that all of my previous transactions were no longer
correctly categorized as well, I threw in the towel.
Cooper was very definite on this one:
What went wrong: Clearly, there was some sort of issue with the conversion
program.
Why it happened: Conversion testing typically gets little to no attention. In
this situation you may have had two completely different development
teams, one for the Windows version, and one for the Mac version of the
program.
How software development teams can prevent it: Perform adequate
testing of the conversion processes prior to release.
Getting at the root causes
Why do such glitches continue to occur with such frequency that they’ve become a
part of my everyday life? For one thing, there’s a lot of legacy code out there, and
not all of it is maintained by following current best practices. But part of the
problem also may be that developers are moving too fast too soon. “It used to be
that upgrades happened every 18 months,” Cooper said. “Now it’s every few weeks
or even every day, and there are more things that can break.”
That’s dangerous, because some issues are more than an inconvenience,
particularly with the Internet of Things, he added. For example, having your smart
thermostat shut down the heat in your home during the coldest day of the year can
be dangerous. What if you’re not home to reboot that thermostat?
While everyone wants to speed up the development pipeline, if you start slow and
follow best practices, you’ll improve software quality, retain more users, and have a
big advantage over your competitors who feel pressured to rush releases and
updates to market.
Do you think the number of software quality issues is on the rise? Or is it just that
the explosion in the number of apps out there makes it seem so? What’s your take
on the state of software quality? Is the need for speed getting ahead of quality in
your organization? I look forward to hearing your stories.
2016 World Quality Report -latest trends in QA & testing
Topics: Performance
Share this article
Show More
Get Report
Get Report
Fail Software failure
app failure
More by Robert L. Mitchell
All open, all the time
The 8 best open-source tools forbuilding microservice apps
The TechBeacon interview
Coding for privacy: Aconversation with TRUSTe's KenOkumura
Article Tags
Robert L. MitchellManaging Editor
TechBeacon
3 articles
Follow @rmitch
Subscribe to TechBeacon
Get fresh whitepapers, reports, casestudies, and articles weekly.
Enter your email address
! SUBSCRIBE
Newest | Oldest | Top Comments
Sign in 2 people listening
Related Articles
Pairs, mobs, and coffeebreaks: How social softwaretesting can benefit your orgMob mentality
Outages happen: 4 ways toprevail over your next bigsystem failureWing and a prayer
Never say die: How QAengineers can becomeperformance testersQA's evolution
5 misguided principles of webperformance (and how torevise them)Not so fast...
Relax! Good encryptionpractices won't affect appperformanceDial it right
How to build performanceinto your user storiesMake it a way of life
Moving targets: Testingsoftware in the age ofmachine learningSo unpredictable
First days as an automationdeveloper? Take my advice onbalancing dev, QALearn the ropes
Joe Colantonio: How to gettesters and developersworking togetherFlock mentality
Plan your tests and releasecriteria the right wayQA takes the lead
5 big changes IT leaders needto know in the DevOps ageAll in on outcomes
9 metrics that can make adifference to today’ssoftware development teamsMeasure what matters!
" # $
Brought to you by
Agile App Dev
DevOps IT Ops
Mobile Performance
Security
Topics TechBeacon
About
Our Contributors
Terms of Use
© 2016 TechBeacon
0 comments
Post comment as...
⋆
⋆
⋆
⋆