OpenAIR Judging Guidance(Version 1.0, revised November 2013)

Core Accessibility.....................................................................................................................2Inactive Images....................................................................................................................................2Active Images.......................................................................................................................................2Hypertext Links....................................................................................................................................2Correct Markup and Parsing................................................................................................................3Skip Links.............................................................................................................................................3Landmark Roles...................................................................................................................................3Headings..............................................................................................................................................4Information in presentation layer........................................................................................................4Contrast...............................................................................................................................................4Keyboard Navigation...........................................................................................................................5Language.............................................................................................................................................5Forms...................................................................................................................................................5Focus Indication...................................................................................................................................6Text Sizing............................................................................................................................................6

Basic Accessibility, Usability and Rally Requirements Judging Guidance............................................7Spelling................................................................................................................................................7Distracting Content..............................................................................................................................7Form Controls......................................................................................................................................7Links.....................................................................................................................................................8Timing..................................................................................................................................................8Pop-Ups (Secondary Window).............................................................................................................8Titles....................................................................................................................................................8Site Map..............................................................................................................................................9Search..................................................................................................................................................9Language Changes...............................................................................................................................9Contact Information..........................................................................................................................10Navigation Links.................................................................................................................................10Knowbility Logo.................................................................................................................................10Last Update Date...............................................................................................................................10High Contrast Mode..........................................................................................................................11

Overall Site Quality Judging Guidance.............................................................................................12Aesthetics..........................................................................................................................................12Exemplary Effort................................................................................................................................12Site Appropriateness.........................................................................................................................12

Advanced Accessibility Judging Guidance.......................................................................................13Data Tables........................................................................................................................................13Video.................................................................................................................................................13Audio.................................................................................................................................................13Alternate Style Sheets.......................................................................................................................14Scripted Features...............................................................................................................................14Fluid Layout.......................................................................................................................................15Responsive Design.............................................................................................................................15Judges Assessment for Inaccessible Content.....................................................................................15

Core Accessibility

Inactive Images

1. Inactive Images (20 points). Use the alt attribute to provide a short text alternative for non-linked (inactive) images. If the image is decorative or redundant, use an empty alt attribute (alt=“”) or integrate the image through CSS instead; if the image conveys information, the image has to be integrated through HTML and the alt-text should convey the same information as the image. If this is not possible with, for example, a chart or graph, identify the function or purpose of the image with the alt attribute and convey the information through an in-line description, a link to a text description, a longdesc attribute, or ARIA-describedby. If the image consists of text, the alt-text should be the same. Each image without appropriate alt-text or lacking an alt attribute is an error.

Tools

Jim Thatcher’s Favelets, JAWS, Web Developer Toolbar, Firebug, WAVE Toolbar

Methodology

If getting different results in different tools disable CSS.

Active Images

2. Active images (20 points). Use the alt attribute to provide a short text alternative for linked (active) images. The text alternative must convey the function of the active element, or the target of the link. Use the alt attribute to provide a text alternative for every active image, including image links, image map hot spots, or input elements of type image. The text alternative must convey the function or purpose of the active element or the target of the link. If the image consists of text, the alt-text should convey the same information and the alt text should be the same or clearer. Each active image without appropriate alt text or lacking an alt attribute is an error.

Tools

Jim Thatcher’s Favelets, JAWS, Web Developer Toolbar, Firebug, WAVE Toolbar,

Methodology

Use tool to determine if alternative text describes function or destination.

Hypertext Links

3. Hypertext Links (20 points). Use concise descriptive language for active elements such as links, menu items, etc. to ensure that each active text element makes sense when it is read out of its immediate context thus do not use link text such as "read more", "more", "click here", etc. Each active element with inadequate text is an error.

Tools

Visual Inspection, JAWS links list,

Methodology

Read the links and make sure that they are adequately descriptive.

Correct Markup and Parsing

4. Correct Markup/Parsing (20 points). Use semantic markup (block quotes, headings, lists, etc.) to properly represent the structure of the document. All elements should have complete start and end tags, be nested according to their specifications, not contain duplicate attributes. In addition, any IDs are unique, except where the specifications allow these features. Each instance of a structural tag used for formatting or content that should use semantic markup or that is not structured to specifications is an error.

Do not use table structure to control page layout. Tables used for purely presentational purposes are assigned an aria presentation role (role="presentation")

Tags used for formatting or content 1) that should use semantic markup or 2) that is not structured to specifications is an error.

Tools

WC3 Validator, Visual Inspection

Methodology

Be lenient with labeling of sections make sure that section is used semantically.

Skip Links

5. Skip links (10 points). Unless there are fewer than 4 links between the top of the page and the start of the main content there must be a “skip to main content” that points to the role=”main” landmark. This link must always be visible or must become visible when it receives focus. Using a skip navigation widget is also permissible provided all options in the widget point to a valid ARIA landmark. The link or widget must be keyboard accessible.

Tools

Browser - test with multiple

Methodology

Make sure destination is <main> element. Make sure that next tab stop is in correct location.

Landmark Roles

6. Landmark roles (10 points). Use ARIA landmark roles to identify regions. All content on the page must be within an appropriate ARIA landmark. Each page that does not meet these requirements is an error. For regions using role="article", each article should have an aria-labelledby referring to the heading of that article. The heading must reside at the start of that article. For roles appearing on more than one region use either aria-label or aria-labelledby to identify the region.

Tools

JAWS, Juicy Studios Accessibility Toolbar, Jim Thatcher Favelets,

Methodology

Make sure that all visible content is inside of a region; make sure that things are nested properly. main, banner and contentinfo only appear once.

Headings

7. Headings for navigation (10 points). Use headings to identify sections and subsections of content. Within the landmark region you must have consistent heading levels. Each page should have at least one H1 heading and heading levels should not skip (H2 then H4). Each incorrect use of headings is an error.

Tools

Wave, Jim Thatcher’s Favelets, JAWS

Methodology

Each sectioning element should result in a correct outline of the page

Information in presentation layer

8. Information in presentation layer (20 points). Ensure that information conveyed through presentation (such as location, font, color, or font attributes such as emphasized or strong) or sound (audio clips or sound files) is also available in text. Make sure color alone is not used to convey content, identify visual elements or distinguish a link from its surrounding text, unless the color contrast between the link and the surrounding text is at least 3:1 and an additional differentiation such as underlining is provided when the link is hovered over or receives focus.

Tools

This one requires manual inspection of the page. You can test contrast of the links in the body of the document using a contract testing tool. The tool included in the internet explorer web accessibility tool bar is a good one. Or the Juicy Studios tool.

Methodology

Look for clues to problems in the text. "Required fields are in red" for example. Tab through all of the links on the page. Do the links stand out from the background when they receive focus. Check contrast of body links using a contrast checking tool. Links used in main navigation areas may not need decoration provided you can tell when you have focus on them.

Contrast

9. Contrast (20 Points). The visual presentation of text and images of text has a contrast ratio of at least 4.5:1, except for a) text or images of text that are part of an inactive user interface component, b) that are purely decorative, c) that are not visible to anyone, d) that are part of a picture that contains significant other visual content or d) that is part of a logo.

Tools

Juicy Studios Contrast Analyzer, Color contrast tool in IE Accessibility toolbar

Methodology

Look for areas where contrast seems to be low visually. Using the Juicy Studios tool you will get a list of all possible combinations. If a combination fails you must confirm that the combination is used on the page. If using the eye-dropper tool make sure that you are getting an accurate color swath. Examples of inactive user interface components include disabled form controls, disabled links, or similar items. Inactive user interface components do not include read only text fields.

Keyboard Navigation

10. Keyboard (20 Points). All functionality in the page must be device independent (accessible with the keyboard only). Keyboard functionality should not require the use of mouse keys, the JAWS cursor, or other similar techniques. Any instance of an operation or a control that does not fully function using only the keyboard results in a score of 0.

Tools

Keyboard, visual inspection. JAWS.

Methodology

Navigate and operate the page using only standard navigation keys (tab, shift+tab, arrows, enter, space, escape). All operable elements should receive focus in a logical sequence. Any buttons or widgets must work with the keyboard in a manner consistent with conventional expectations for the widget.

If using JAWS, you can emulate the experience of a non-JAWS user by pressing insert+z twice quickly to disable the virtual cursor for all applications. Note that if you press insert+z only once, the disabled state will apply only to the browser and the virtual cursor will be re-enabled if you move focus outside the browser and then back to the browser.

Note that while testing keyboard operability, you can test #13 (focus indication) and #37 (scripted features) at the same time.

Language

11. Language (20 Points). "The language of the page is identified in the HTML element, using the lang attribute with a two-character code value set to the current language of the page ("en" for English, for example). Any failure to identify the natural language in a page results in a score of 0".

Tools

Web Developer Toolbar, IE Accessibility Toolbar, WAVE

Methodology

Review page properties every page must have a correct language attribute that matches the natural language of the page.

Forms

12. Forms (20 points). For native html form controls with a single label use the label element. If the label is not visible use title. For non-native controls use aria-labelledby for visible labels and aria-label for non-visible labels. Use fieldset/legend to group radio buttons and check boxes. Use the HTML5 required attribute for required fields and/or aria-required property. Use ARIA markup for error handling including the aria-invalid state on fields which have errors. Use the alert or alertdialog roles for announcing the errors.

Tools

Jim Thatcher’s favelets, IE web accessibility tool bar, WAVE, JAWS*

Methodology

Using one of the tools look for any errors of missing form field labels or incorrect id/for attribute matching. For fields without a visible text label check the title attribute using either code inspection or the JAWS form field list. If JAWS misreports any form field it is an error. Check that groups of radio buttons or check boxes are grouped with fieldset. For non-native elements ARIA radio group may be used.

Focus Indication

13. Focus indication (20 points). There must be a clear visual indication when an object receives focus. This focus indication needs to be at least as strong as the default browser focus indication. Each instance of an object with no focus indication is an error.

Tools

Keyboard and Mouse, Inspect 32, AccPro

Methodology

Visual Inspection see if focus goes away. Do this at the same time as keyboard.

Text Sizing

14. Text Sizing. (20 Points). Text can be resized independent of general browser magnification. Text reflows properly when resized. Any instance where either the text cannot be resized or where the text fails to reflow properly when resized is an error.

Tools

No Squint, Browser Text Size controls

Methodology

Look for overlap, broken layout, If using control+ in Firefox you must check text zoom only first. Do not use browser zoom for this. You must increase text size to 200%. Preferred is no-squint because you can control the magnification better.

Basic Accessibility, Usability and Rally Requirements Judging GuidanceIn this section with the exception of spelling once you have found one error you can stop checking for the item. The site receives 0 points for the item and you can stop looking at that aspect. Most of these items can be tested while testing the home page for other items.

Spelling

15. Spelling. Spelling is correct. 1 point deducted per spelling error.

Tools

MS Word or Open Office Spell Check, JAWS

Methodology

Choose several pages at random, copy into a word processing program and run spell check. Listen to a few pages with JAWS listen for unusual pronunciations - note you should be used to using JAWS frequently to recognize the usual JAWS mispronunciations vs the misspellings.

Distracting Content

16. Distracting Content. Avoid distracting content such as moving text or blinking or flashing areas. Ensure webpages do not contain anything that flashes more than three times in any one second. Any animation or moving content must stop by itself in 3 seconds or must provide a user controllable method to end the animation or movement. Any violation result is a score of 0.

Tools

Visual Inspection, stop watch.

Methodology

Look for moving content. Is there a pause button? Does it stop automatically after 3 seconds or less. If it is an animate gif you can determine the exact time by looking at the frame rate and number of frames played in Photoshop. If the mechanism to stop the animation is not keyboard accessible score 0 and deduct from keyboard access.

Form Controls

17. Form Controls. All standard HTML native form controls function as they would normally be expected to. Any standard HTML native form control that does not function as expected results in a score of 0.

Tools

Web Browser

Methodology

If there are any forms test the operation of all form controls. If you find any unexpected behavior award 0 points. Particularly look for form controls like a drop down list that responds to an onChange event.

Links

18. Links. All hyperlinks function properly. Any link not functioning improperly (e.g. wrong page, page not found, etc.), results in a score of 0.

Tools

Web Browser, JAWS Links List, Link Checker add on for Firefox (There are several of these)

Methodology

Using either a link checking plug in or manual testing check for any broken links. The link text should also describe where the link goes so that you can judge if it is taking you to the correct location.

Timing

19. Timing. Except for real-time events where the time limit is absolutely required, if a page or application has a time limit, the user is given options to turn off, adjust, or extend that time limit as often as needed to complete the task. Failure to provide such flexibility results in a score of 0.

Tools

Web Browser, Patience

Methodology

Look for likely places where there might be a time out. If there is a secure area of the site that requires a log in this is a good place to check. Walk away from the browser for a while, 10 min should be more than enough, and see if the page times out. Determine if you get a warning and if you can request more time or adjust the time. If you can adjust the time or request more time make sure that the process is fully keyboard and screen reader accessible. Make sure that the warning receives focus when it occurs.

Pop-Ups (Secondary Window)

20. Pop-ups. If content open opens in a new browser window or a new tab the user is informed that the content opens in a new window or tab and a method is provided for closing the window or tab. Any failure yields score of 0.

Tools

Web Browser, JAWS

Methodology

Follow the links, if any content opens in a new window and the link does not indicate that this will happen then award 0 points. Warning must be available to the screen reader through alt text or actual text in the link. If indication is in Title - 0 points. This does not apply to JavaScript Alerts. If using JAWS (or another screen reader) fail if JAWS announces that a new window is opening if the indication was not presented first. This can be done while testing other aspects of links like link text. Overlays are also not considered pop-ups.

Titles

21. Titles. All pages and frames have titles that properly describe their topic or purpose. Any page or frame title that is not appropriately described results in a score of 0.

Tools

Web Browser, JAWS, Wave

Methodology

Check each page title, is it unique to the site. Is it descriptive of the page that you are on. Test this while testing other as you move through the site.

Site Map

22. Site Map. Users can access all significant pages of the site using a well-structured site map or the main navigation menu. If no site map or its equivalent is provided, score is 0

Tools

Web Browser

Methodology

Look for a link to a site map. If it exists does it link to all significant pages on the site. If not 0 points. If no site map are all significant pages included in the main navigation for the site. If not 0 points.

Search

23. Search. An accessible search engine is available, and easy to find. Must have <label> or title attribute and role = search for points. No search or inaccessible search - score is 0

Tools

Web browser

Methodology

While checking the home page look for a site search, if it is not there - 0 points. If not indicated by role=search - 0 points. If no label or title - 0 points.

Language Changes

24. Language Changes. The human language of each passage or phrase in the content can be programmatically determined, except for proper names, technical terms, words of indeterminate language, and words or phrases that have become part of the vernacular of the immediately surrounding text. Identify language changes within a page. Any failure is score of 0.

Tools

Web browser, JAWS, Wave

Methodology

While checking the site for other items, if you observe any foreign words or phases, or a section of content in a foreign language check to see if there is a language tag. If using JAWS listen for really bad pronunciation. If this is a commonly used foreign word or phrase that has been incorporated into English it does not need to be inside of a language tag. If you are judging a French page - contact Denis.

Contact Information

25. Contact Information. Email and/or contact information is provided and easy to find using the address element. Contact information that is not available or hard to find results in a score of 0.

Tools

Web Browser

Methodology

On the home page, is there a link to contact information in the main navigation or the footer. Is it inside of the address element. If not score 0. If the main navigation or the footer is inconsistent across the site score 0.

Navigation Links

26. Navigation links. All repetitive navigation links are text (not images). If images are used for repeated navigation links, score is 0. This requirement does not apply to logos that are linked to the site home page.

Tools

Web Browser

Methodology

Using the web developer tool bar, or the IE accessibility tool bar, or wave disable images on the home page - if the main navigation is not present score 0. If using JAWS if the main navigation is announced as an image link score 0.

Knowbility Logo

27. Knowbility Logo. There must be a Knowbility logo and link to the Knowbility site that is easy to find. With no Knowbility logo or no link to Knowbility score is 0.

Tools

Web browser, JAWS images list

Methodology

While testing the site for other items check for a linked Knowbility Logo image. It may say Knowbility, Knowbility Logo, or even Knowbility AIR. The image must link to the Knowbility site, either the home page or the AIR page. Typical places to find this are the page footer, the about page, or an accessibility page. If found elsewhere make a judgment as to whether it was easy to find.

Last Update Date

28. Last Update Date. At least the home page must have an update date. Not providing an update date on the home page results in a score of 0.

Tools

Web browser

Methodology

While checking the home page look for a update date on the page if it's not there score 0.

High Contrast Mode

29. High Contrast Mode Support. All content remains visible and usable when in High Contrast Mode. Failure to support high contrast mode results in a score of 0.

Tools

Operating System

Methodology

Using your operating systems high contrast settings. In Windows this can be found in the ease of access center and in Mac it is found in system preferences accessibility settings. I recommend checking a few options. If the site does not respond to the settings or if content disappears then score 0. This only needs to be checked on the home page. Test focus indicators as well.

Overall Site Quality Judging Guidance

Aesthetics

30. Aesthetics (up to 20 points). How enjoyable, overall, is the experience of using and navigating this site? Is the overall look and feel attractive? Do colors, graphics, backgrounds, icons, fonts, etc., contribute to the ability to easily read and understand the content. Are graphics and text descriptors (including ALT attributes) meaningful and purposeful? Does the site provide a coherent auditory experience for people using screen readers and talking browsers? Points are awarded at the discretion of the Judge.

Tools

Browser, Screen Reader, Screen Magnifying Software (possibly, Dragon Naturally Speaking as well).

Methodology

This is a pure taste and judgment area. Reading several pages with a screen reader is strongly encouraged, as is using a screen magnifier if you have one available. The questions to ask yourself when determining the number of points to award is how pleasant was it to use the site, can you find things easily, does the reading order make sense etc.

Exemplary Effort

31. Exemplary effort (up to 20 points)! Judges prerogative. For clever things that provide for accessibility. Not to reward design or aesthetics - specifically effort in accessibility. Judge will explain in comments.

Tools

Your own good judgment.

Methodology

Determine if the developers have really done something innovative. This is where we as judges learned something from the sites developers. It should be difficult to get points in this area.

Site Appropriateness

32. Site Appropriateness (up to 20 points). How well does this site reflect the nature of the organization it represents? Consider language, images and graphics, and other content. Is the organization's mission clear? Is the site structured to allow for maintenance and growth? JUDGED BY THE NPO OR ARTIST CLIENT. Due back 2 working days after site has been submitted.

Tools

None

Methodology

The lead judge will collect the responses from Knowbility staff and add these points to the completed forms from the other judges. Judges should leave this section blank.

Advanced Accessibility Judging Guidance

Data Tables

33. Data Tables (10 points). Use the CAPTION element and/or the SUMMARY attribute to describe data tables. Use the TH element to unambiguously identify all row and column headers. For complex data tables, associate data cells with the appropriate headers using either HEADERS/ID combination or the SCOPE attribute. Each instance of missing or incorrect table markup is an error and results in a loss of 5 points.

Tools

WAVE, Web Accessibility Tool Bar for IE, Jim Thatcher’s Favelets, Screen Reader.

Methodology

Using a table inspection tool verify that all table headers are in TH elements, verify that the headers are in the same table as the data. If judging a complex table with headers on the diagonal or more than one column of row headers verify that headers and ids are being used correctly. If judging a table with multiple rows of column headers verify that colspan and scope are being used correctly. Every error is a loss of 5 points so if you have a table with three columns without headers then score 0.

Video

34. Video (20 points). For video with soundtrack, provide synchronized captions. Provide an html text description for video without sound. In addition, provide synchronized audio description if the video cannot by understood from the soundtrack alone. If not synchronized, audio description must be provided as text with the video link. Each instance of inadequate accommodation is an error.

Tools

Browser, Keyboard

Methodology

Play the video, are captions present or can they be turned on. Are the captions synchronized. Is there video content that requires audio description to be understood. If so, is audio description present or can it be enabled (separate versions are permissible). Also permissible for audio description is a separate text document. To receive credit the video player must be accessible.

Audio

35. Audio (10 points). Provide html text transcripts for audio files. Speakers must be identified. Each place where the text transcript substantially differs from the audio is an error.

Tools

Browser

Methodology

If there are any audio only files a transcript must be linked next to the link to the audio file. The transcript must accurately reflect the content of the audio file. Read the transcript as you listen to the audio to check for accuracy. If you are using a screen reader you will need to work from memory.

Alternate Style Sheets

36. Alternate Style Sheets (10 points). Provide at least two style sheets in addition to the default that can be conveniently selected by users, for example, to change font size, contrast, color schemes, printing, or displaying on small devices. Any page where the layout fails with any alternate style sheet or where the selection process fails is an error. 35. Alternate Style Sheets (10 points). Provide at least two style sheets in addition to the default that can be conveniently selected by users, for example, to change font size, contrast, color schemes, printing, or displaying on small devices. Any page where the layout fails with any alternate style sheet or where the selection process fails is an error.

Tools

Browser

Methodology

Check for a mechanism for switching style sheets. The mechanism should be present in contrast greater than or equal to 4.5:1. If you can change style sheets is the method used accessible. If not count an error. Check to make sure that the new style remains persistent as you move through the site if you must set the style sheet multiple times then count as an error. If there is a color switching style sheet make sure that the user cannot select the same color for both foreground and background or that there is a simple method to allow recovery from what will become and invisible page.

Scripted Features

37. Scripted Features (20 Points). All scripted functionality meets the following requirements:

a) Scripted controls are not used to replace semantic html elements like buttons and links.b) All interactive widgets are keyboard accessible using standard design patternsc) Name, role, state, value and properties are managed appropriately. Where applicable ARIA best

practices are used to code special widgets including tab panels, accordion menus and modal windows.

d) Notification of changing content is provided appropriately

Tools

Browser, keyboard, screen reader, Jim Thatcher’s Favelets

Methodology

Inspect all interactive features - the submission form should list anything that they have scripted. Make sure that they have not created an interface element that should have been coded using standard HTML elements. Test for keyboard operability. Using a screen reader or a browser plug in make sure that name state and role are managed appropriately. If using live regions make sure that the notice of the change is appropriate.

Fluid Layout

38. Fluid Layout supports reflow with browser magnification (20 points). Using browser magnification in at least two browsers, increase the magnification level to 200%. If content reflows to prevent the necessity of using a left-right scroll bar to access content and all content is available, 20 points. If screen does not reflow 0 points.38. Fluid Layout supports reflow with browser magnification (20 points). Using browser magnification in at least two browsers, increase the magnification level to 200%. If content reflows to prevent the necessity of using a left-right scroll bar to access content and all content is available, 20 points. If screen does not reflow 0 points.

Tools

Browser

Methodology

Increase magnification to 200% with the browser set to 800 pixels wide content should reflow without the need of a left right scroll bar.

Responsive Design

39. Responsive Design (20 points). Site is implemented using responsive design techniques to facilitate appropriate presentation on desktop/laptop, tablet and mobile devices. To receive points must support at least 3 device types e.g. smart phone, tablet, and laptop/desktop. All functionality present for smartphone and tablet presentations must work with touch screen interface.

Tools

Browser, cell phone, tablet.

Methodology

Contact developers to see what type of devices were used to test for accessibility by them. Try to match devices at least in terms of general os.

Judges Assessment for Inaccessible Content

40. Judges Adjustment for Inaccessible Content. Judges may subtract points (judges discretion) beyond giving "no credit" for a checkpoint for grievous use of inaccessible coding or content. Judge in consultation with one other judge is required to document instances and state reasons for points deducted. Examples: Major use of inaccessible multimedia, objects (Flash, JAVA), navigation. [Enter as a positive number!]

Tools

Judgment

Methodology

If you identify other accessibility issues that are not specifically covered by the judging form that significantly impact the accessibility of the site for one or more groups of people this is the place to make the deduction. Additionally if accessibility barriers would prevent someone from using a major feature of the site you may choose to deduct additional points here. Must be done in consultation with another judges. Recommend contacting the lead judge.