the 7 minute accessibility assessment and app rating system

The 7 Minute Accessibility Assessment

& App Rating System 

AIDAN TIERNEY ! @AIDANA11Y

"

‘Is it accessible?’   #

No budget No time

WCAG What?

No budget No time

7 minute assessment

WCAG What?

5 star rating system

An accessibility rating system non-specialists will understand

⋆ ⋆ ⋆ ⋆ ⋆

How long does it take your users to rate your apps?

%

Rapid vs. comprehensive

full assessment 7 minute assessment

output ratinglist of every issue

rating few word summary

no issue list

skill needed intermediate advanced or expert

time days minutes

cost significant minimal

7 minute assessment output is only a rating,

with a few sentence summary.

No list of issues.

Things you'll need

&Advanced skills in WCAG conformance testing:

• screen reader testing experience • colour contrast testing • assessing severity or user impact of issues

& Understand rating system and severity/user impact scales 

& Ability to imagine another user's experience

& Login details and test data, if necessary  

The 7 minute assessment procedure

Launch app 

Explore - push, trigger, grab

Media

Identify up to 4 test screens and a core task

Run Android Accessibility Scanner (if applicable - it's not for Health)

Screen reader: read from top

Listen for: name, role, state

Note any issues

Repeat for all test screens

Screen reader: complete a core task, trigger errors

Send to workstation: Screenshots & Scanner results 

On workstation: colour contrast

Can be skipped on Android (Scanner finds these)

Review issues: severity & impact

Issue  Severity

Profile button missing accessible name

Colour contrast issues

Severity & User Impact

Blocks: total? task? with workaround?

Annoyances

Trivial and technical non-conformance

User context influences impact

Severity User impact of issue  Examples

1 Crash or total blockCannot log in

Inaccessible CAPTCHA No screens reads in screen reader 

2 Cannot complete core task

Unlabelled items - no contextNo captions on video

Items don't work in screen reader Severe colour contrast/use colour

3

User may experience difficulty, confusion, irritation but issue

would not prevent a user from completing a task 

Unlabelled control, clear in context  Items read twice

Error presentment non-fatal issues Minor colour contrast

4Minor inconvenience or

Technical non-conformance with no user impact 

Spelling mistakes in alt text or captions (meaning preserved)

Review issues: severity & impact

Issue  Severity

Profile button missing accessible name 3

Colour contrast issues 3

Assign rating, Write very brief summary (no defect list)

You'll also send a 'key' to rating and severity scales

⋆ ⋆ ⋆ ⋆ ⋆

5 Star Accessibility Rating System

⋆ ⋆ ⋆ ⋆ ⋆

(⋆ ⋆ ⋆ ⋆ ⋆

⋆ ⋆ ⋆ ⋆

)⋆ ⋆ ⋆

*⋆ ⋆

⋆

RatingLevel of

accessibility Tolerance for issues

5 star Very high

No Severity 1 or 2.Some minor impact issues may exist.

No issues prevent user completing a core task. Low incidence of issues.

4 star High

No Severity 1 or 2.Some minor impact issues may exist.

No issues prevent user completing core task. Users could get through tasks, despite annoyances.

3 star Moderate

No Severity 1. May contain Severity 2.Issues may prevent users completing non-core tasks.

Users experience many annoyances, confusion.Lack of attention to detail across the application. 

2 star LowIssues will prevent users completing core tasks.

Poor overall accessibility quality.    

1 star Very low Severe issues and overall inattention to accessibility.

You could modify the degree of tolerance for your own scale.

The point is to have a rating scale that a non-specialist will understand.

Assign rating, write summary

⋆ ⋆ ⋆ ⋆ ⋆

"Five stars. Very accessible.

Unlabelled button on home screen.

Minor colour contrast issues"

Demo: iOS Health App

Lab: let's try it out

Review issues: severity & impact

Issue  Severity

Assign rating, Write very brief summary (no defect list)

You'll also send a 'key' to rating and severity scales

⋆ ⋆ ⋆ ⋆ ⋆

Lastly

Not a substitute for full testing

Subjective, yes, but imagine other users

No list of issues

Requires accessibility experience

Can use ratings to summarize full assessments too

“What do you think?

Could you use this?

What would you change?”

+

,

The 7 Minute Accessibility Assessment

& App Rating System 

AIDAN TIERNEY ! @AIDANA11Y

"