Take this app and... Jul 8, 2016

5 reasons why users are ready totrash your app

Every morning, I launch my NPR news app, press play, and—nothing happens. Each

time, I must go to another screen to re-select my local NPR news station, return to

the main screen, and press play again. Only then does the broadcast start. I

shouldn’t have to do that. The app is supposed to remember my local station.

Nevertheless, the workaround has become part of my morning routine because I

am willing to put up with it to hear the news.

But when an ABC News app began interrupting me with notices that it had stopped

functioning—even when I wasn’t running it—I swiped it into the trash without a

second thought. My life is full of software issues that somehow must have survived

the quality assurance gauntlet, and the incidence of such annoyances only seems to

be increasing.

2016 World Quality Report -latest trends in QA & testing

What about the apps you or your team have built and deployed? Are they simply

being tolerated by irritated users? How many times did your app get swiped into the

trash, without you even knowing it? And what should you do about it?

The software quality conundrum

Ask a typical consumer whether the state of software quality has improved and you

may not like what you hear. Over the last six months, I’ve personally had to deal

with a barrage of software-related issues, and I’m not talking about apps from mom-

and-pop development shops. Some problems I've experienced come from apps built

by some of Silicon Valley’s biggest software companies —the ones held up as the

true innovators in software development.

In my day job, I speak with software quality experts, attend conferences, and talk to

developers and software quality experts in leading-edge businesses who sing the

praises of using DevOps, agile, and continuous delivery pipelines to get better-

quality software to market faster. But at home, someone is always struggling with

one application glitch or another. And it’s not just mobile and desktop apps. The

Internet of Things has introduced its own series of problems, from my home

security cameras that mistake noise for motion and issue false alerts to the smart

thermostat that just can’t seem to maintain its connection to my router.

“It’s not just you,” said Michael Cooper, chief quality officer at Healthcare IT

Leaders and Run Consultants, after hearing my long litany of software issues. “The

issues you describe seem to have happened to a lot of people. It’s a real problem.”

Cooper, who has led QA teams at HP, T-Mobile USA, and FICO, feels the burn. He

had to deal with a programmable thermostat that shut off his home heat last winter

due to a software glitch and a Windows 10 upgrade that corrupted all of the data on

his hard drive.

I asked him to weigh in on five of the most frustrating software-related issues I have

faced recently. I wanted to know what went wrong, why these issues keep cropping

up with such alarming frequency, and what software developers can do to prevent

similar issues in their own applications.

1. The mobile OS upgrade from hell

Not long after I installed Google’s Android Lollipop update, things began to go

horribly wrong. At first, just a few apps began to respond slowly or crash. But the

situation gradually worsened over the next few days, until my phone became

practically unusable. Wiping it and reinstalling the software helped initially, but

then the glitches would just begin anew.

At one point, the Android phone dialer app froze in a Windows-like blue screen of

death; at other times the screen went black after I initiated a call, making it

impossible to hang up. The last straw was when my phone crashed in the middle of

an important conference call. I powered it back up, only to see this dialog:

“Optimizing app 1 of 135…” More than a half-hour later, the phone finally came back

online. But by then I had missed a call with my boss, and I lost even more time after

deciding to wipe and rebuild the phone—again.

For business users such as me, this is the downside of IT consumerization. The new

features in Lollipop, while nice to have, did not deliver any quantifiable business

value. And they introduced risks. What I value most is stability and reliability, not

cool new features. I suspect I'm not alone in this preference.

For this reason, I initially resisted the nagging OS upgrade messages. But the carrier

said there was no way to turn those reminders off, so I finally gave in. Only after the

problems began did I realize that there was no going back, no “restore to last known

good state.”

My carrier blamed the mobile operating system and the phone manufacturer for my

woes. Its solution: Buy a new phone, just 14 months into a 24-month contract for a

piece of business equipment that cost $700. Ouch. After several calls to tech

support and three visits to the carrier’s store, I ended up dumping the phone.

Here’s Cooper’s take:

What went wrong: Most likely there were some incompatibilities with the

Android code and the Qualcomm chipset in my device. Clearing the cache

might have resolved the issues. Unfortunately, that advice came too late for

me.

Why it happened: Google, device manufacturers, and wireless carriers don’t

test Android upgrades on every device that’s out there. Older devices are

rarely tested for compatibility with new versions of Android, and most

device makers add proprietary code to Android, increasing the potential for

upgrade-related issues. Even with new devices, Cooper says, “There is often

a rush to market, and testing is limited.”

How software development teams can prevent it: For Android’s software

developers, it’s important to understand which devices are most popular and

test the top 20 percent. That should cover more than 80 percent of your

users. The best ways to get this level of test coverage are through

crowdsourced testing, mobile test automation, and the use of a cloud-based

mobile testing lab.

Device makers should collaborate with OS makers to test their new device lineup

on the latest and upcoming OS versions, Cooper added. Carriers and device makers

can reduce problems by limiting customization of the Android operating system.

Apple doesn’t allow nonstandard versions of IoS, which is why such issues occur less

often with iPhones, he said.

As for users, they should wait to upgrade until bugs have been worked out for their

phone, Cooper advised, and wipe the cache immediately after an upgrade.

2. The online banking upgrade that threw easeof use under the bus

I loved my bank’s old bill-paying system for its simplicity. Once I had set up my

payees, I brought up the list, typed in the amount to pay and the payment date in

two little boxes next to each vendor’s name, clicked the “pay” button at the bottom

of the window, and I was done.

With the new bill-pay system, I must click on each biller name individually to open a

pop-up window, then click the buttons on a calculator image to enter the dollar

amount, press the “enter” button, and then open another window to set the

payment date before clicking “submit.” To pay the next biller, I must return to the

main bill-pay screen and start the process all over again.

Here's what Cooper had to say:

What went wrong: Most likely this is due to poor design, with no

consideration of its impact on the user.

Why it happened: It could be due to a lack of user experience (UX) training

and, most likely, the business performed no UX testing.

How software development teams can prevent it: Hire professional UX

designers and perform UX testing with feedback. The most customer-centric

companies, such as Intuit and Apple, are known for UX testing and use of

focus groups.

3. E-billing Catch-22: After choosing e-billingover paper, I get neither

When I signed up to receive e-bills from one payee in my bank’s online bill pay

system, the paper bills stopped arriving, but the e-bills didn’t show up in the system.

When I attempted to turn off the e-billing option, the software complained: “You

must receive at least one online bill before e-billing can be turned off.” I then

attempted to delete the biller, only to be blocked again. The system’s response:

“Biller can’t be removed until online e-billing is turned off.”

Cooper had some ideas:

What went wrong: It’s possible that my spam filter blocked my first e-bill, or

corrupted data might have prevented the system from sending the e-bill to

the correct email address.

Why it happened: It is hard to anticipate all of the issues that can go wrong

with e-billing. There are usually several systems involved, and all must be

integration-tested from end to end, including the actual sending of the bills

and the process of switching from e-billing to paper billing and back. Often in

a testing cycle, teams don’t budget enough time to run several billing cycles,

including the mailing of real paper bills and the e-mailing and verification of

e-bills.

How software development teams can prevent it: End-to-end testing must

include switching between e-billing and paper billing options and back.

Amdocs, which tested billing for T-Mobile, created a logical date in its system

so it could test multiple billing cycles with representative synthetic data in

order to avoid exactly this type of situation, Cooper said.

4. Gaming software rejects payment, dumpsuser data

When my nephew received a popular PC game for Christmas, my brother-in-law

went online to register and pay for it. He filled out the profile information for

himself and his young son, entered his credit card information, and promptly

received a “payment rejected” error. When he returned to the registration screen to

try again, all of his personal profile and billing data had been discarded. He repeated

the process using a different credit card, with the same result. As it turned out, both

credit cards were fine. But this particular user ended up re-entering his profile data

three times before the issue was finally resolved, and then only after making a call

to technical support.

Cooper suggested the following reasons for my brother-in-law's hassle:

What went wrong: To process a payment, the vendor must make a call to a

credit card processor. The credit card transaction must pass several fraud

checks, and all of the information needs to match before it can be processed.

It must be clear to the user that, for example, the address entered must

exactly match the billing address on file with the credit card company. A

temporary outage in the connection to the payment system also could have

led to the problem. While the credit card information must disappear for

security reasons if the request is rejected, other data that the user has

entered could have been retained.

Why it happened: Many companies stop testing before making an actual

payment because they don’t want to spend the money during the test,

Cooper said.

How software development teams can prevent it: Get test accounts from

the credit card processing company so you’re not actually getting charged

during testing. Also, be careful not to trigger fraud protection during credit

card testing. End-to-end testing is key, and tests need to be run against all

major payment types you plan to accept.

5. Software upgrade corrupts user data

My migration from Windows to the Mac version of a popular personal finance

program turned into a data conversion debacle. The Mac version could not read the

data file created in the Windows format without a conversion step. When that

process failed, the vendor offered to do it for me, and I gladly uploaded my file. But

when the Mac-ready file came back, "memorized transactions"—a feature that

recognizes a transaction and automatically assigns it to the proper reporting

category—didn’t remember what went where. The categories came over, but the

logic on when to apply them didn’t. Tech support was unable to help, and I resigned

myself to manually recreating all of my memorized transactions as I created new

entries. But when I realized that all of my previous transactions were no longer

correctly categorized as well, I threw in the towel.

Cooper was very definite on this one:

What went wrong: Clearly, there was some sort of issue with the conversion

program.

Why it happened: Conversion testing typically gets little to no attention. In

this situation you may have had two completely different development

teams, one for the Windows version, and one for the Mac version of the

program.

How software development teams can prevent it: Perform adequate

testing of the conversion processes prior to release.

Getting at the root causes

Why do such glitches continue to occur with such frequency that they’ve become a

part of my everyday life? For one thing, there’s a lot of legacy code out there, and

not all of it is maintained by following current best practices. But part of the

problem also may be that developers are moving too fast too soon. “It used to be

that upgrades happened every 18 months,” Cooper said. “Now it’s every few weeks

or even every day, and there are more things that can break.”

That’s dangerous, because some issues are more than an inconvenience,

particularly with the Internet of Things, he added. For example, having your smart

thermostat shut down the heat in your home during the coldest day of the year can

be dangerous. What if you’re not home to reboot that thermostat?

While everyone wants to speed up the development pipeline, if you start slow and

follow best practices, you’ll improve software quality, retain more users, and have a

big advantage over your competitors who feel pressured to rush releases and

updates to market.

Do you think the number of software quality issues is on the rise? Or is it just that

the explosion in the number of apps out there makes it seem so? What’s your take

on the state of software quality? Is the need for speed getting ahead of quality in

your organization? I look forward to hearing your stories.

2016 World Quality Report -latest trends in QA & testing

Topics: Performance

Share this article

Show More

Get Report

Get Report

Fail Software failure

app failure

More by Robert L. Mitchell

All open, all the time

The 8 best open-source tools forbuilding microservice apps

The TechBeacon interview

Coding for privacy: Aconversation with TRUSTe's KenOkumura

Article Tags

Robert L. MitchellManaging Editor

TechBeacon

3 articles

Follow @rmitch

Subscribe to TechBeacon

Get fresh whitepapers, reports, casestudies, and articles weekly.

Enter your email address

! SUBSCRIBE

Newest | Oldest | Top Comments

Sign in 2 people listening

Related Articles

Pairs, mobs, and coffeebreaks: How social softwaretesting can benefit your orgMob mentality

Outages happen: 4 ways toprevail over your next bigsystem failureWing and a prayer

Never say die: How QAengineers can becomeperformance testersQA's evolution

5 misguided principles of webperformance (and how torevise them)Not so fast...

Relax! Good encryptionpractices won't affect appperformanceDial it right

How to build performanceinto your user storiesMake it a way of life

Moving targets: Testingsoftware in the age ofmachine learningSo unpredictable

First days as an automationdeveloper? Take my advice onbalancing dev, QALearn the ropes

Joe Colantonio: How to gettesters and developersworking togetherFlock mentality

Plan your tests and releasecriteria the right wayQA takes the lead

5 big changes IT leaders needto know in the DevOps ageAll in on outcomes

9 metrics that can make adifference to today’ssoftware development teamsMeasure what matters!

" # $

Brought to you by

Agile App Dev

DevOps IT Ops

Mobile Performance

Security

Topics TechBeacon

About

Our Contributors

Terms of Use

© 2016 TechBeacon

0 comments

Post comment as...

⋆

⋆

⋆

⋆