Title Slide

Computer says …Adventures of an analog person in a digital world

Les Hatton

Professor of Forensic Software Engineering,CISM, Kingston UniversityL.Hatton@kingston.ac.uk

Version 1.1: 03/Jun/2010

Professorial lecture: June 9th 2010, CISM, Kingston University. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

More about software than you ever wanted to know“Computer says” and other little problemsInterfaces from hell and other illiteraciesSecurity ….. ?

Overview

3

Its in everything

The amount of software in consumer electronic products is currently doubling about every 18 months. Some examples:-

• TV, videos, DVD players, MIDI systems

• Answering machines, mobile phones• Washing machines, microwaves, central

heating controllers, burglar and fire alarms• Cars, trains• Planes• Nuclear Reactor control systems• If you can read this you should really begin to worry.

4

Its like a recipe …

Software is a set of instructions just like a recipe. Like cooking, it uses the following:-

• Containers of different sizes• Ingredients, (different kinds of number)• Instructions for mixing the ingredients in the

containers

5

… but its much bigger

If written out normally on sheets of paper• NASA Shuttle software: 3,000 pages• Flight control software: 10,000 pages• In your car: 100,000 pages• In your video and media players: 1,000,000

pages (i.e. a pile about 100 metres high).

6

It looks like this …

Just as recipes are written in many languages, programs are also. This is an example of one commonly used language in internet applications:-

foreach (%rqpairs = split(/[&=]/, $ENV{“QUERY_STRING”)) {

tr/+/ /;

s/%(..)/pack(“c”,hex($1))/ge; }

while (($key,$value) = each %rqpairs) {

print “<LI>$key = $value`n”; }

print “</UL>End of variables.</BODY></HTML>“;

exit 0; }

blah blah blah ...

7

and one tiny mistake ...

1.5 million more lines this way, (~600 km. at this scale)

switch( message ){case INCOMING_MESSAGE:

if ( sending_switch == OUT_OF_SERVICE ){

if ( ring_write_buffer == EMPTY )send_in_service_to_smm(3B);

elsebreak; /* Whoops ! */

}process_incoming_message(); /* skipped */break;

...}

1.5 million more lines this way

8

in 30,000 pages …

… caused the whole US long-distance telephone network to crash, (January 1990).

So its quite hard to get it right. In fact its impossible but that is no excuse for what follows …

9

Some of it is exceedingly good …(Space Shuttle software)

Images copyright NASA and USAF, Space Shuttle Atlantis 14-May-2010

10

… but most of it is not(all within 90 minutes at Heathrow, 11-May-2010)

Departures

“This system is rubbish” (departures official)

Check -in

Departures lounge On the plane

11

Check-in problems happen to me quite a lot

• Feb 2010, print off boarding cards online• Departures won’t let me in because they can’t read it.• SAS can’t issue another boarding card because

I already have one

• I generously offer to lie on the runway until they sort it out.• SAS duty manager gives me written one if I promise “I am

me”.

• Departures won’t let me in because I now have two passes until they find person who refused the first one.

• I point out that its lucky I’m not Bertrand Russell. This comment is wasted. Note to self: no more jokes at check-in

12

This sort of thing is surprisingly common

24.5 million XP crashes per dayhttp://www.pcmag.com/article2/0,4149,1210067,

00.asp

5% of Windows Computers crash more than twice a day

http://www.nytimes.com/2003/07/25/technology/25SOFT.html

13

… and not everything that goes up, comes down in one piece

Ariane 5

14

Many large projects are late or never appear at all or don’t do what they

are supposed to

• NHS “Connecting for Health” – budget over 16 billion pounds

• Child Support Agency• Passport Office• Benefit Office• C-Nomis (2009) (Ministry of Justice and Home Office) –

“Nobody sure how 161million pounds had been spent”• Transport Direct cycle route planner (2009). This

absorbed 2.7 million pounds but failed to replace a public site with far greater coverage and functionality (cyclestreets.net) which cost around 6,000 pounds.

15

Honesty is the best policy

16

More about software than you ever wanted to know“Computer says” and other little problemsInterfaces from hell and other illiteraciesSecurity ….. ?

Overview

17

The strange case of the FAX machine and the telephone sex line

• Receive telephone bill for FAX machine containing two 20m05s calls to a telephone sex line at 4am.

• I call BT. “Computer says” its correct• I ‘elevate priority’ and provide FAX logs. BT

says it will investigate• Final demand received• I call BT. “Computer says” its correct• Notice of removal of service arrives• I call BT and invite them to take me to court• BT calls me. “Computer says” its correct but

they will let me off “if I don’t do it again”

6 weeks

18

10/April/2006: Malaysian man gets $218 trillion phone bill. (Associated Press).

• Telekom Malaysia gave 10 days to pay. They later decided it was “a little excessive”.

There’s something about phone companies …

19

Why I hate digital television

• Freeview box 1: Download yes / no ?• Freeview box 2: Crashed every 3 hours• Freeview box 3: Crashed every 7 hours• Freeview box 4: Crashes every few hours (I don’t care

any more)

• Note Hatton’s Law: Television quality is inversely proportional to the number of channels on offer.

20

More about software than you ever wanted to know“Computer says” and other little problemsInterfaces from hell and other illiteraciesSecurity ….. ?

Overview

21

Purchased 5 years ago and faultless ever since.Staggeringly simple and intuitively obvious interface which has never required the instructions to be consulted.

A tale of two alarm clocks:Alarm clock 1

22

A tale of two alarm clocks –the sublime

Hour

Minute

Time set

Alarm set

AlarmOn/off

Drowse

23

Purchased 1 year ago and frequently resets itselfStaggeringly complex and intuitively non-obvious interface.

A tale of two alarm clocks:Alarm Clock 2

24

A tale of two alarm clocks –Interface from hell

Mode- ARGH !

Morebuttons

Desk

Randomnumbers

25

Time settingPress MODE button THREE times. Then press SELECT and SET repeatedly.

Alarm settingPress MODE button TWO times. Then follow time setting.

Hourly chime on/offPress SELECT and MODE button simultaneously

Alarm onPress DATE and SELECT button simultaneously

Stop watch / lap counterPress MODE button ONCE and then beg for mercy as clock gibbers with entertaining series of random beeps.

Turning alarm offHurl out of window after standing on clock screaming.

A tale of two alarm clocks:Interface from hell

26

Even the mighty Apple produces awfulinterfaces

iPhoto ™, ready for action

27

The delights of iPhoto

What does this mean ? Note the lack of grouping and causality.

28

What they really mean …

This photo has already been imported to your computer

Would you …• Like to import it again because you don’t trust me, (and I don’t blame you after I drew one upside-down and made it a different size)• Like to skip it because against all the odds, you do trust me• Like to import this and any other duplicates so I don’t have to ask again• Like to skip this and any other duplicates so I don’t have to ask again• Like to forget the whole thing and maybe practise the trombone

Cracking cheese Gromet Cracking

cheese Gromet

29

“Assumption is the mother of all screw-ups” *

My first medical system experience, (a medical records system which each night backed itself up with the message … )

Storing

* Wethern’s law of suspended judgement

30

“Assumption is the mother of all screw-ups”

Unfortunately, it was delivered in the Netherlands which after suitable translation yields …

Jamming

31

Flight management system

An example from real life, Airbus A320 AF319, 25/8/88, (Mellor (1994)):-• MAN PITCH TRIM ONLY, followed in quick succession by ...• Fault in right main landing gear• Fault in electrical flight control system computer 2• Fault in alternate ground spoilers 1-2-3-5• Fault in left pitch control green hydraulic circuit• Loss of attitude protection• Fault in Air Data System 2• Autopilot 2 shown as engaged when it was disengaged• LAVATORY SMOKE

32

An Airbus having a bad day

A Tarom airlines Airbus which performed an uncontrolled dive,climb, roll and spin near Orly in 1995 due to ‘a fault in the automatic pilot’.The plane landed safely, a tribute to the pilots’ skill.

33

More Airbus’ having bad days

16th October 2008“The Qantas A330-300 incident where the plane nose-dived hundreds of feet injuring 70 passengers was caused by a computer fault that shut down the autopilot, reported Australian Transport Safety Bureau.”

27th December 2008“The autopilot in another Qantas A330-300 suddenly disconnected en route to Singapore from Perth. Pilots returned to Perth. ATSB said this was similar to the previous incident and both would be treated together”

34

Some examples …• 23/Jan/2004. The new 1.2 billion pound London

Underground Tube smartcard was reporting a ‘code 24’, (negative pre-paid balance !), condition stopping many travellers from getting on their trains.

Incomprehensible error messages ...

Courtesy Dr. Andreas Breu

and …

and …

35

The great local bar disaster

Programmers effort:-

System over-stressed ...

Symptom: The author’s local bar was unable to dispense beer.

Translation into English:-

The printer has run out of paper(Try explaining this to a thirsty native)

and …

36

The inscrutable ...

• This appears on the Flight Management System of a McDonnell-Douglas MD-11, (Drury (1997))

• The pilot also noted “The airplane [computer system] manuals were written as though by creatures from another planet”.

Programmers effort:-

Button push ignored

and …

37

Vodafone’s attempt to upgrade my Mobile WiFi connection

and …

38

Credit card PIN readers

Consider this from the point of view of the blind

39

… and it goes on …

11/08/2008• Failures in new NHS computer system have

meant hundreds of suspected cancer sufferers in London have their operations cancelled.

• People in contact with MRSA could not be contacted

• Many appointments lost.http://www.bbc.co.uk/1/hi/england/7555077.htm

19/09/2006• Failures in new NHS computer system have led

to 110 ‘major incidents’ in 4 months.http://www.dailymail.co.uk/

40

Banks and payrolls …• 08/07/2008. Westpac in Australia issued all payroll and

direct debits twice.http://blogs.spectrum.ieee.org/riskfactor/2008/07/westpac_bank_glitch_causes_pro.html

• 11/08/2008. Swansea Council in Wales had to abandon a GBP 819,000 project with CAP Gemini after it increased to GBP 8,000,000 after continual problems.

http://blogs.spectrum.ieee.org/riskfactor/2008/08/

… and on …

41

Cars …• 05/02/2010. Toyota Prius braking problem. (causing one

second lag in application of brakes.)• 06/02/2005. Whole string of problems, shaking Mercedes,

Ford that bakes back seat passengers …http://www.nytimes.com/2005/02/06/automobiles/06AUTO.html

• 26/10/2004. BMW disables dynamic stability control and ABS. Two police drivers vindicated after investigation.

http://www.daserste.de/plusminus/beitrag.asp?iid=254

• 14/04/2004. Ford is recalling 363,440 of its 2001-2003 Ford Escape vehicles due to software problems in power-train causing engine stalling.

Detroit News

… and on …

42

The author’s favourites ...

• “Entire Montgomery Ward warehouse goes missing for 3 years”.• An error in the input program lost the warehouse in Redding,

California. The staff didn’t like to say anything because they thought they had lost their jobs but the pay-chequescontinued to arrive.

• “Computer program drives Arab to sexual exhaustion”.• (Computer talk 1/2/88 - an unfortunate bug in the

gentleman’s harem organisation program led to his downfall).

• The US Navy’s wonderful intelligent torpedo ...

43

More about software than you ever wanted to knowComputer says and other little problemsInterfaces from hell and other illiteraciesSecurity ….. ?

Overview

44

Beware of e-mail

Event # occurrencesReceived messages 836,106

Rejected as non-compliant - 618,000Silently discarded as junk - 217,275

Content filtered spam/scam - 13

Delivered (to 8 domains) 818 (0.098%)

(Root dictionary attacks) 160

A week in the life of a mail-server, November 2009

45

Knock, knock its your internet burglar …

Imagine your internet-connected PC is a house.

• Every 2 minutes somebody peers through the window

• Every 3 minutes, somebody climbs over the wall and walks round the house

• Every 80 minutes, somebody tries your doors and windows

• Every 3 hours, somebody will try to break the door down with a sledgehammer.

46

Thousands of PCs and PDAs controlled by remote criminal operators

Mariposa botnet (Spain, Mar 2010) had details of 800,000 people gleaned from 12.7 million machines.Waledac (US, Feb 2010) (hundreds of thousands of PCs) used for sending hundreds of millions of spam messages each day.Lethic (Jan, 2010), Mega-D (Nov 2009), Torpig (May, 2009), McColo (Nov, 2008) all taken down after efforts by security reseachers.

About every 3 weeks the PC of a personal acquaintance is compromised.

Botnets

47

Organisations have an appalling record …Mar 2010, Barnet council lost 9,000 children’s recordsAug 2008, Zurich insurance lose about 600,000 records, (came to light in Mar, 2010).Jan, 2010, Ladbrokes lose 4.5 million customer records.Oct 2008, 25 million child benefit records lost, (this included 350 on witness protection scheme).2005-2008, HMRC reported 7 other significant data loss incidents.Dec 2007, hundreds of thousands of patient records from nine NHS trusts went missing. The NHS also routinely shares patient data with local councils.In 2008, the MoD lost almost as many records as the NHS.Dec 2007, Department of Transport admits three major data loss incidents including those of 3 million learner drivers.Aug 2008, HSBC lost an entire server with 159,000 customer records.

Data loss, (many, many examples)

48

Computer says ...

•There are many, many benefits, but ...• Security is getting more challenging, so beware,• Over-ambition is the enemy of quality systems,• Just because we can automate something doesn’t mean we

should, (Government 3.0),• Any database system is no better than its data and the

resources we spend validating it,• It dehumanises many people with incomprehensible interfaces,

inconsistent behaviour and unquestioning trust,• We can’t make perfect systems but we could do much better.

•High quality IT systems are at the heart of an efficient economy. Its time we learned these lessons.

49

The essence of Forensics …

“Those who cannot remember the past are condemned to repeat it”. George Santayana, (1863-1952)

Notes for this talk are available at:-http://www.leshatton.org/

“On Ko Chi Shin” – “Studying the old promotes a better understanding of the new”. Ancient Japanese proverb.