enriching web information scent for blind users

Enriching Web Information Scent for Blind Users

Markel Vigo1, Barbara Leporini2, and Fabio Paternò2

1 University of the Basque Country

2 Human Interfaces in Information Systems

1 Laboratory of HCI for Special Needs

2 Italian National Research Council

Hypothesis: annotating links with the accessibility level of the page where they point would increase user orientation

“Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.

”Detecting and notifying users about barriers improves user orientation”- Harper et al.

Goal: use of accessibility assessment results in web navigation scenarios

Information Scent:

Thus, we aim at enriching information scent using accessibility assessment results for screen reader users.

Enriching Web Information Scent for Blind Users ACM ASSETS 2009

1. Introduction

1. Visual or textual cues provided on a Web site to suggest what information its links may contain.2. The perceived usefulness of a page based on such information.

Hypothesis: annotating links with the accessibility level of the page that the link point to increases user orientation

“Visually impaired users need to be warned of obstacles because their reliance on cues is higher than for sighted users”- Goble et al.

”Detecting and notifying users about barriers improves user orientation”- Harper et al.

Goal: deployment of accessibility assessment results in navigation scenarios

Information Scent:

Thus, we aim at enriching information scent using accessibility assessment results


1. Introduction

1. Visual or textual cues provided on a Web site to suggest what information it or its links may contain.2. The perceived usefulness of a page based on such information.

not accessible

highly accessible

fairly accessible

…calls for


2. Challenges

• automatic evaluation due to efficiency needs- based on guideline review tools- be aware of tool limitations- make assumptions, take risks

• user-tailored assessment- current assessment techniques address all user groups- adaptive evaluation and measurement- quantitative scores for accuracy and discrimination power

Goal: annotation of links with accessibility assessment results in navigation scenarios.


3. User-Tailored Assessment: Evaluation

• Web Accessibility: subset of WCAG 1.0 - following the classification by Brajnik for the BW method

- developed the Accessibility Checker for Blind users (ACB)

- ACB checks lack of tags, attributes and appropriate combination of them

Accessibility is measured in terms of conformance to web guidelines for blind users.

• Web Usability: Usability Guidelines for the Blind (UGB) - 4 principles: structure and arrangement, content appropriateness,

multimodal output, consistency - automatic guideline review tool: Magenta- Magenta checks adequate content of tags and attributes, arrangement of

headings or shortcuts


3. User-Tailored Assessment: Evaluation

• Resolving guideline set conflicts/overlap - Addressed- Addressed but not implemented

- Complementary

- Contradictory

Accessibility is measured in terms of conformance to web guidelines for blind users.

ACB

Magenta

www.foo.com

Accessibility report

Usabilityreport

Dependencies solver

Exclusively accessibility issues

Exclusively usability issues


3. User-Tailored Assessment: Measurement

• Calculate failure-rates for each test case, earl:TestCase• Using aggregation methods • Considering issue typology:- automatic issues (earl:automatic) yield earl:passed or earl:fail- recommendations- semi-automatic issues (earl:semiAuto)

ACB

Magenta

www.foo.com

Accessibility report

Usabilityreport

Dependencies solver

Exclusively accessibility issues

Exclusively usability issues

Metrics Calculation Component

accesibility score


3. User-Tailored Assessment: Measurement

Traditional aggregation:

€

E=W1E1 +.. +WiEi +.. +WnEn

Logic Scores Preferences:

€

E= W1E1ρ( d)

+.. +WiEiρ( d)

+.. +WnEnρ(d) ⎛

⎝ ⎜

⎞ ⎠ ⎟1ρ( d)

where W: weights and E: evaluation results

where ρ(d) are values selected upon the required logical relationship between evaluation results.

d=0 conjunction 0< d <0.5 quasiconjunction: simultaneity in satisfying all the evaluationsd=0.5 arithmetic mean0.5< d <1 quasidisjunction: penalizes only when all evaluations are not satisfied d=1 disjunction

Only intermediate values are applied


3. User-Tailored Assessment: Measurement• Example: a checkpoint implements 3 test cases

test cases

checkpoints

T1 T2 T3

guidelines

0.3

0.25

- strong quasiconjuction among earl:automatic

T1earl:automatic= 1

T2earl:automatic= 0.25

T3earl:semi-automatic= 0

- medium quasidisjunction among earl:semi-automatic

0.25

1 0.25 0

final score

- weak quasiconjunction is applied at checkpoint level

?

- between guidelines, mean value


4. User-testing• Preliminary evaluation for annotated links • Experimental settings- 16 experienced blind users. Age M=43, sd=11- JAWS on Internet Explorer- Remote usability testing

- Log analysis of interaction sequence, timing, keyboard and mouse actions

- Post-task forms and post-test questionnaire

• Two tasks- Browsing by navigating - Searching by navigating


4. User-testing: browsing by navigating

• Definition: casual/aimless browsing deciding at each step where to go next

• Goal: observe users with no/vague target in mind

• Two sites with 10 links- Top ten search results for “Pisa” and “Firenze” keywords

- Results were heterogeneous links wrt topic,

- Following a pattern: wikipedia, local university, soccer team and so on

- One site was manually annotated with accessibility scores and relevance scores

- Relevance based on ranking {very relevant, relevant, medium, low, irrelevant}

- 5 min free browsing. They had to write a report about what they learned


4. Results for browsing by navigating

• In the page without annotations,- 9 users proceeded sequentially. Kendall τ=[0.8-1.0] at most p<0.03

• In the page with annotations,- Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05

- None followed the sequence of most accessible links

- None followed the path based on relevance

- Some proceeded dichotomously

- However, when aggregating accessibility scores of visited pages, 7 points over the median are obtained

- This, can be interpreted as if the users browsed within the subset of more accessible pages according to random/preference criteria


4. User-testing: searching by navigating

• Definition: look for a target by sequentially deciding at each step where to go next

• Goal: observe users with a specific target in mind

• Two sites with 10 links- Top ten search results for “Accommodations in Pisa”

- Results were homogeneous links wrt topic

- One site was manually annotated with accessibility scores

- Relevance was not considered

- Two tasks: (1) given a telephone number (2) address

- They had to write down the name of the hotel


4. Results for searching by navigating

• In the page without annotations,- Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05

- One user proceeded inversely. Kendal τ=-1.0 at most p<0.02

• In the page with annotations,- Only 2 users proceeded sequentially. Kendall τ=1.0 at most p<0.05

- One user followed the most accessible path. Kendal τ=1.0 at most p<0.02

- Again, when aggregating accessibility scores of visited pages 6 points over the median are obtained


5. Results: gathered informal comments • 8 users appreciated accessibility scores in links• The suitability of scores was more controversial

…some were satisfied: - “values adequately reflect accessibility level”- “scores are useful”, “scores are interesting”- “accessibility scores seem correct”- “navigation is better if scores are included”

…while other were not: - “strange validation”- “scores are not very coherent”

…other changed their minds: - “I’m doubtful about accessibility criteria” “links with accessibility scores are

useful”- “scores don’t convey the actual difference in accessibility level” “scores make

navigation smoother and more instinctive” - “scores seem random””I missed accessibility scores in this task”


5. Results: post-test questionnaire

• “Scores are useful for the browsing task”

1 2 3 4 5

0

2

4

6

8

10

• “Scores are useful for the searching task”

• “Scores are correlated with actual accessibility level in browsing task”

• “Scores are correlated with actual accessibility level in searching task”

1 2 3 4 5

0

2

4

6

8

10

1 2 3 4 5

0

2

4

6

8

10

1 2 3 4 5

0

2

4

6

8

10

• 5 point Likert-scale {1: totally disagree – 5: totally agree}


6. Conclusions: scores

• Most users state that scores are useful to a certain extent- Although there is not an agreement- “The perception of accessibility depends on each user and their

particular computer settings”

• There is no agreement on the type of scores they prefer- 50% for qualitative and quantitative

• Considering the informal comments it seems that the annotation technique prevails over scores


6. Conclusions: annotation technique

• In the searching scenario users do search within the subset of most accessible links • In the browsing scenario users change paradigm- From sequential browsing to random in the subset of most

accessible links- Subjective scores are more balanced than in the searching

scenario

• When directly enquired, users state that accessibility annotations would be useful in those scenarios where the topic of the linked pages would be similar


6. Conclusions: annotation technique • It seems that annotation technique would better fit in an scenario where:

- They are browsing casually - and topics of linked pages are similar

• For instance on the leaf nodes of a web directory


Markel Vigo1, Barbara Leporini2, and Fabio Paternò2

1 University of the Basque Country

2 Human Interfaces in Information Systems

1 Laboratory of HCI for Special Needs

2 Italian National Research Council

Questions?

enriching web information scent for blind users

Technology

web accessibility

enriching information

web usability

web guidelines

web site

blind usersacm assets2009

accessibility level

accessibility checker