characterizing web heuristics - simon fraser...

APPLIED THEORY SUMMARYr Proposes a framework for analyzing the

characteristics of heuristics for Web designr Demonstrates the value of the framework by

analyzing three existing sets of guidelines

Characterizing Web HeuristicsMENNO DE JONG AND THEA VAN DER GEEST

INTRODUCTION

The World Wide Web has grown rapidly and hasbecome a widely used means of communication.Yet little is known about “what works” in Website design. Designers of any communication sys-

tem, product, or policy try to apply the best availableknowledge from fields such as linguistics, social and cog-nitive psychology, and human factors research. But theresearch findings of these fields are often contradictory;furthermore, it is questionable whether findings from re-search in these fields can be generalized to the designproblem at hand. They rarely are so specific that they canhelp the designer in making decisions during a particularproject.

Web designers, even more than designers of othercommunication products, have difficulties deciding aboutfunctions, audiences, content, tone, words, images, sound,and so forth (van der Geest, in press). Compared withdesigners in traditional paper media, who present informa-tion through text and graphics, the Web designer has aneven wider range of presentation modes and variables tochoose from, options that interact and mutually influenceeffectiveness. Additionally, the medium is so novel thatalmost any solution is a nonstandard solution, especiallybecause until now, most attention about Web design hasfocused on the technical development rather than on itscharacter as a means of communication. It is not surprisingthat many Web designers resort to Web design guidelinesthat capture the knowledge and experience of others.

A quick Internet search using the keyword Web designproduces 754,904 hits. A search within those hits using thekeywords design guidelines, design principles, designrules, design criteria, or heuristics produces a total of 384hits. These 384 hits form a broad collection of guidelines onthe technology and tools that make the Web run (forexample, Javascript or HTML), galleries of images andsounds, user interface design principles, page and sitedesign guidelines, and style guides for the Web pages ofspecific companies and organizations. Imagine the poorWeb designer who wants to make sensible choices fromthis jumble of guidelines. Which of these to choose?

The term Web heuristics is used to refer to all the sets

of process guides, principles, criteria, tips and tricks, andguidelines that are available to support Web designers. Aheuristic is a discovery aid. It helps “problem solvers” (andwe consider Web designers as such) to identify a problem,to get an overview of the range of options to choose from,and to make sensible choices. We assume that designersare using some kind of heuristic if they use an aid orprocedure that helps them to choose in a well-reasonedway a particular design approach or a particular designoption, or if it helps them to assess the qualities of anoption. Heuristics may be prescriptive or instructive, andthey usually take the form of lists of questions, principles,or checkpoints. In our use of the term heuristic, we followcommon terminology such as heuristic evaluation—that is,evaluation on the basis of a defined set of principles.

Heuristics help designers by directing their attentionand promoting exploration of the range of options from aparticular perspective. That capability, however, impliesthat heuristics limit other possible areas of attention orperspectives (Byard 1991). Therefore, designers shouldhave a feeling for the underlying assumptions of the heu-ristics they are using.

Our goal for this article is to make Web designers moreaware of the qualities of heuristics by presenting a frame-work for analyzing the characteristics of heuristics. Thisframework is meant to support Web designers in choosingamong alternative heuristics. We hope that better knowl-edge of the backgrounds, potentials, and limitations ofheuristics will contribute to the professional expertise inthe field. Our second goal is to make those who developand present heuristics more aware of the information theirusers need. Thus, we have tried to increase the usability ofheuristics.

A FRAMEWORK FOR CHARACTERIZING HEURISTICSWhereas heuristics are being developed and presented tohelp Web designers focus their attention on various aspectsof Web site quality, our framework seeks to help Web

Manuscript received 29 January 2000; revised 21 March 2000;accepted 22 March 2000.

Third Quarter 2000 • TechnicalCOMMUNICATION 311

designers asses the usefulness and usability of the heuris-tics themselves. The framework consists of four main cat-egories. The first deals with the information covered by theheuristics: Which aspects of Web site quality do the heu-ristics focus on? The second concerns the validity of heu-ristics: What do we know about their background andjustification? The third concerns the presentation format:How are heuristics presented to the Web designer? And thelast category deals with the—often implicit—assumptionsabout the way heuristics are to be used in practice. Anoverview of the framework is presented in Appendix 1.

INFORMATION COVERED BY THE HEURISTICSA good starting question about heuristics is what the set ofheuristics is all about. What topics are covered by theguidelines, questions, or criteria? To what kinds of Websites are the heuristics to be applied?

SpecificityFirst is the issue of specificity: To what kinds of Web

sites and site characteristics do the heuristics apply? Heu-ristics may be designed for general use, thus suggestingthat they are a suitable tool for all kinds of Web sites andsite characteristics. But there may also be two kinds oflimitations in scope.

1. Heuristics may be genre-specific, focusing on aparticular type of Web site such as a digital store or aproduct information site.

2. They may be feature-specific, focusing on certainsite characteristics such as navigation, graphics and lay-out, or accessibility for people with disabilities.Both general and specific heuristics may be useful tools forWeb designers. It is important, however, to know theirlimitations.

ExhaustivenessGiven the degree of specificity, to what extent can the

heuristics be expected to cover everything of interest? Heu-ristics may range from an exhaustive selection of items,representing at least the most important aspects of thedomain, to a more or less arbitrary selection of items.Although the idea of heuristics fully covering everything ofinterest in a certain domain is hard to achieve in practice,the reverse—an entirely random selection of items—is farfrom desirable.

There are various ways of reducing arbitrariness. If theheuristics are based on empirical research, there may be arationale in the underlying empirical data. Nielsen (1994, p.30), for instance, based his heuristics on a factor analysis of249 problems frequently found in usability tests of softwareinterfaces. The claim of the resulting heuristics may be thatthey cover most of the usability problems that occur inpractice. Oliver, Wilkinson, and Bennett (1997) used an-

other empirical strategy to account for their selection ofitems in their information quality heuristics. They asked apanel of experienced Internet users to rate the importanceof an exhaustive list of quality indicators and selected theindicators with the highest importance ratings.

Another strategy to reduce arbitrariness could be asystematical approach. Heuristics may be designed to sys-tematically explore every aspect of a Web site from acertain perspective. In the case of genre-specific heuristics,they may be based on a selection of usage scenarios; in thecase of feature-specific heuristics, overall concepts may besystematically broken down into a set of lower-level crite-ria. Whatever strategy is used, Web designers should un-derstand the rationale behind the heuristics.

VALIDITY OF THE HEURISTICSHeuristics may look quite impressive as they are presentedto the practitioner. But Web designers should not selectheuristics just on first impression. Four questions may helpto judge the validity of heuristics—that is, the extent towhich they can be expected to contribute to effective Website design.

FoundationsWhat kind of support is offered for the various heuris-

tic items? Heuristics can be distinguished on the basis oftheir foundations.

r Standards-based heuristics These heuristics arebased on official, agreed-on rules for Web design,and compliance with them may lead to some kind ofcertification. In this respect, standards-based heuris-tics are valid by definition, although agreed-on rulesdo not necessarily have a strong relationship to theusability and effectiveness of Web sites. Some of theheuristics focusing on accessibility for people withdisabilities come close to being standards. The WebContent Accessibility Guidelines by W3C (1999), forinstance, encourage Webmasters to mention thelevel of conformance to the standards. Based on thesame guidelines, an evaluation tool called Bobbywas developed to enable Webmasters to automati-cally evaluate their site’s accessibility and—in thecase of a positive result—include a “Bobby Ap-proved” icon on their site (CAST 1999).

r Theory-based heuristics These heuristics areadapted from general, well-known, and acceptedtheories. As the research into the use and effective-ness of Web sites is just emerging, many of thesetheories might be adapted from other relevant ar-eas—for example, human-computer interaction, textcomprehension, rhetoric, or visual design. One ofthe few examples of theory-based heuristics are theCognitive Engineering Principles by Gerhardt-Powals

APPLIED THEORYCharacterizing Web Heuristics de Jong and van der Geest

312 TechnicalCOMMUNICATION • Third Quarter 2000

(1996), focusing on “cognitive friendly” user inter-faces, that seem applicable to the World Wide Webas well.

r Research-based heuristics These heuristics arebased on specific research that puts the usability andeffectiveness of Web sites to the test. One can thinkof the aggregated results of a series of usabilitytests—as Nielsen (1994) did for user interfaces—orthe results of experimental research comparing theeffects of design variations. Evans (2000) describesthe difficult process of translating the results of em-pirical research into a set of heuristics. Again, manyof the heuristics available for Web design originatefrom research with more or less similar communica-tion products—for example, hypertext, multimedia,and user interfaces.

r Practitioners’ heuristics These heuristics refer toguidelines that are not supported by standards, the-ory, or research, but instead reflect the views andexperiences of professional Web designers or thedirections given in handbooks on Web design. Manyof the heuristics mentioned throughout this articleare in fact practitioners’ heuristics.It is not easy to draw the line between the types of

heuristics characterized here. Often, heuristics are partlybased on theories and research, with the remainder drawnfrom practitioners’ views. Heuristics from all four catego-ries may contain valuable clues for improving the quality ofWeb sites. Yet it is important that the users of heuristics beinformed about the rationale behind and empirical supportfor the heuristic items. The relationship between specificheuristic items and their motivation or evidence should bemuch tighter than is the case with many existing heuristics.

NoveltyIn addition to the foundations of heuristic items, it is

also important to consider the way in which they evokeand expand the design knowledge of the practitioner usingthe heuristics. On the one hand, heuristics may have highnovelty value, drawing attention to new and surprisinginsights on Web design. These appear to be meant forexperienced Web designers, who have internalized thebasics of Web design. On the other hand, heuristics mayhave low novelty value, primarily giving an overview of theexisting body of knowledge regarding Web design. Thesemay be very informative for beginning designers or serveas a mnemonic device for practitioners, helping them toconsider all relevant aspects of Web design. Both noveland familiar sets of heuristics can be useful in practice.

Room for interpretationTo what extent do the benefits of heuristics depend on

the idiosyncratic intuitions, insights, and experiences of the

individuals who use them? This question, of course, closelyrelates to the traditional methodological concept of reliabil-ity—that is, the extent to which the results are stable. Someheuristics are more or less mechanistic, and different usersare most likely to arrive at the same conclusions. Some caneven be applied automatically: Web designers need onlytype in a URL and the computer applies the heuristics,analyzes the site, and provides feedback—like WebSAT(NIST 1999; Scholtz, Laskowski and Downey 1998) and thepreviously mentioned Bobby (CAST 1999). On the otherhand, many heuristics are expert-mediated. They are meantto focus Web designers’ attention on certain aspects, butthey cannot guarantee at all that different professionals willcome to the same conclusions about these aspects.

Two items from the Web-site Design Audit of theUniversity at Buffalo (W-SDA 1997) may illustrate the dif-ference between more or less mechanistic and typicallyexpert-mediated heuristic items.

r “Can users change information they have entered ina form before they have submitted it?” This is a moreor less mechanistic item. It is easy for Web designersto check whether a site meets this criterion, and theresult of that check is likely to be quite unambigu-ous.

r “Are multimedia, animation, and graphics used onlywhen necessary?” This item requires an expert’sjudgment. Different Web designers may have differ-ent views about the necessity of multimedia, anima-tion, and graphics in a particular site, and may there-fore come to different conclusions about a site.An advantage of mechanistic heuristics is their reliabil-

ity compared with expert-mediated heuristics, but there isalso an important drawback. The need for unambiguousand measurable items may impose serious constraints onthe aspects of site quality that can be addressed, and henceon the validity of the heuristics. So there may be a tradeoffbetween the reliability and the validity of heuristics.

Validation researchA shortcut for avoiding some of the problems men-

tioned above may be found in validation research to backup the heuristics. The validation studies should demon-strate the benefits of the heuristics for professional design-ers. The most convincing proof of benefit would be seenwhen a study demonstrates that a site designed with a setof heuristics is better than one designed without heuristics.Such a study was conducted in the domain of interfacedesign by Gerhardt-Powals (1996). She compared an inter-face designed according to her Cognitive Engineering Prin-ciples with two alternative interfaces that were designedwithout explicit reference to the principles. The cognitivelyengineered interface proved to be superior in use situa-tions, thus supporting the validity of her heuristics.

APPLIED THEORYCharacterizing Web Heuristicsde Jong and van der Geest


Another way to validate heuristics is by comparingtheir yield with the yield of other forms of evaluation.Nielsen (1994, pp. 36–55), for example, compared theresults of a heuristic evaluation by 11 usability specialistswith the problems found in a small-scale usability test (withfour users). The heuristic evaluation revealed more than 80percent of the 21 user problems detected in the usabilitytest and overlooked only 4 user problems. However, Niels-en’s study was flawed because the number of usability testparticipants and usability experts was not balanced. Inanother study, heuristic evaluation made a less favorableimpression (Desurvire 1994). The heuristic evaluation bythe experts in that study revealed fewer than half of theobserved user problems, missing serious problems moreoften than minor annoyances. A problem with this type ofvalidation study is that the criterion for success, overlap ofdetected problems, is debatable. Must heuristic evaluationresults and results of usability testing always overlap? Itcould well be that different methods help to detect differentproblems.

All in all, little is known yet about the validity ofexisting heuristics for Web design. Many of the availableheuristics appear to make sense, but Web designers arerarely provided with enough information about the back-ground and proven benefits of heuristics. General justifica-tions are given where specific evidence for particular itemsis needed—as for example in “The information used tocreate W-SDA has been collected from various sourcesincluding traditional sources for Human Computer Interac-tion Guidelines and Internet sites devoted to Web-site de-sign guidelines” (W-SDA 1999). Validation research into themerits and restrictions of particular heuristics should, in ourview, be placed higher on the research agenda.

PRESENTATION FORMAT OF THE HEURISTICSSo far, our questions about the heuristics have dealt withtheir content. But their presentation format, at the level ofboth the whole set and the individual items, may also be animportant factor for successful use. Our inventory of heu-ristics showed that they are presented in many differentforms, defined by structure, formulation, type of answers,and level.

StructureSome sets of heuristics seem to have a random organiza-tion; others are meaningfully structured. A meaningfulstructure should help a Web designer to view a site from anoverall perspective and to “read between the lines.” Topicsthat are not exactly covered by one of the heuristic itemscan be detected all the same if the Web designer compre-hends the overall intention of the heuristics. A meaningfulstructure is also important if practitioners are expected tointernalize the heuristics. Dividing heuristics into sections is

a fruitful way of creating a meaningful structure. The Webheuristics of Human Factors International (1999), for in-stance, are divided into sections on color, wording, andlayout. However, within the sections, the items of theseheuristics sometimes seem to be ordered randomly, per-haps because of arbitrary selection.

Formulation of itemsBasically, there are three commonly used sentence struc-tures. Heuristics can be formulated as:

r Instructions (“Use color and highlighting sparingly.”)r Questions (“Is the contrast between the background

and the foreground high?”)r Requirements (“The system should always keep us-

ers informed about what is going on, through appro-priate feedback within reasonable time.”)Little is known yet about the effect that formulation has

on the use and yield of heuristics. There might be a relationwith the stage of a Web design process in which theheuristics are applied: Instructions seem to be most appro-priate for the design stage; questions, for the evaluationstage; and requirements may take an intermediate position.

Types of answersA distinction can be made between open and closed heu-ristic items. Typical examples of open items are “Who arethe important users?” and “What is their purpose for ac-cessing the site?” Closed items are often worded in a “yes-no-n/a” format, and laid out as a checklist. Their formsuggests that items can be straightforwardly answered bythe Web designer, but that is not always the case (forexample, “Are multimedia, animation and graphics usedonly when necessary?”). Again, little is known about thepros and cons of closed versus open heuristic items.

Level of heuristicsWright (1985) differentiates low-level and high-level guide-lines. High-level items present Web designers with possibleproblem areas in a Web site, for which they must find asolution—for example, “Make sure the graphic used (dec-orative, representational, organizational, or explanative),and accompanying text matches the kinds of activities youwant viewers to do with them” (Ameritech 1996). Low-levelitems present Web designers with specific design guide-lines rather than with the problems they are intended tosolve—for example, “Use short sentences” (Human FactorsInternational 1999).

According to Wright, both kinds of items have theirown weaknesses. Low-level items suffer, for instance, frombeing too normative and prescriptive, suggesting that Websites must always be designed in a certain way, and thusdisregarding the many fruitful exceptions to the rules. Amultitude of low-level items are needed to cover one as-



pect of Web design, thus reducing the usability of theheuristics. In addition, it is impossible to formulate low-level heuristics to cover every aspect of a particular Website. Finally, low-level heuristics make it hard for Webdesigners to “read between the lines.” High-level items, onthe other hand, leave much to the interpretation and in-sights of the Web designer using them, and hardly presentWeb designers with specific design options for usable andeffective Web sites.

With regard to the presentation format, the structureand the level of the heuristics are especially important. Ingeneral, it seems desirable that heuristic items be presentedin a meaningful structure, enabling the Web designer togain an overview and to judge a site in line with the spiritof the heuristics. In this line of thought, it may be fruitful tohave a combination of high-level and low-level guidelines,in which the low-level items can function to generate ideas,suggest solutions, or serve as examples for more generalhigh-level items. With respect to the two remaining is-sues—that is, the formulation and openness of the items—many variations are seen in practice, but we can as yet only

speculate about the effects they might have on the usabilityof heuristics.

USE OF THE HEURISTICSThe way heuristics are used or are meant to be used—in factthe usability of heuristics—is hardly addressed in the litera-ture. The usability of heuristics is a crucial success factor thatshould have a major impact on the way heuristics are de-signed and presented. Successful use of heuristics can beassured only if they are geared to realistic use situations.

An interesting example of Web heuristics with usabilityflaws is the Web-site Design Audit (W-SDA 1999), of whichFigure 1a and Figure 1b show two successive screens. Theheuristics take the form of “yes/no” questions, and theexplanations for the items are hidden behind hyperlinks.Throughout the set of heuristics, both a “yes” and a “no”answer to heuristic items may indicate a violation of acertain design rule. So there are at least two serious usabil-ity problems here.

1. The heuristics force Web designers to constantlyclick between heuristic items and explanations.

Figure 1a. Heuristics with yes/no answer format (W-SDA 1997).

Figure 1b. Explanation of No-answer to the last question, “Is the third person used for directions?” (W-SDA 1997).



2. The “ yes” and “ no” answers do not correspondunambiguously to positive and problematic site charac-teristics.

Besides the usability problems, the recommendation inFigure 1b, of course, also reveals a major validity problem.The advice to use the third person in directions is contraryto what we know about effective instructions.

Phase in the design processThe first issue of context of use is the phase in the designprocess in which the heuristics are meant to be employed.A distinction can be made between planning-oriented andevaluation-oriented heuristics. Planning-oriented heuristicsare used to generate global or specific requirements duringthe actual design and production of a Web site; evaluation-oriented heuristics are for evaluating and improving a draftor an existing Web site.

Planning-oriented heuristics should facilitate designersin switching between two activities that are hard to com-bine—consulting the heuristics and actually building thesite. Three solutions can be considered:

r The heuristics may aim at internal representation,that is, at being memorized and remembered easily.

r The heuristics may be process-oriented and thus in-terfere less with actually building the site.

r The heuristics may be structured in a way that corre-sponds to specific design activities—instead of a fea-ture-based organization like text readability, visual-ization, or navigation, the heuristics may focus onclearly identifiable parts of a Web site, such as homepage, forms, or menus.Evaluation-oriented heuristics can be used for assess-

ing and improving the quality of Web sites ranging fromearly designs and paper prototypes to Web sites that arealready online. The heuristics may be more elaborate, andmay even take the form of a long checklist. The feature-based organization described above might be very fruitfulin evaluation-oriented heuristics, as it forces evaluators to

view a Web site from entirely different perspectives thanthe ones that were dominant during planning and produc-tion.

Focus of supportHeuristics can be process-oriented and product-oriented.The focus of support for most of the heuristics we haveseen was on the product—the heuristics contained charac-teristics of effective and user-friendly Web sites. Otherheuristics combine a number of process recommenda-tions—concerning the planning phase, the testing, and themaintenance of a Web site—with product guidelines (seeIBM 1999; van der Geest, in press). Especially in the case ofplanning-oriented heuristics, process guidelines may be afruitful way to support Web designers.

Function in the design processHeuristics can have troubleshooting and verifying func-tions, a distinction that can be found in the literature ondocument evaluation (see de Jong and Schellens 1997). Allthe heuristics we studied have a troubleshooting func-tion—that is, they are meant to support the detection anddiagnosis of problem areas in a Web site. Some heuristicshave an additional verifying function. They can be used tocertify whether Web sites or Web site characteristics meetan explicit or implicit criterion. Examples of heuristics withan extra verifying function are the aforementioned Bobby(CAST 1999) and the Web content accessibility guidelinesof W3C (1999).

Alexander and Tate (1999b, p. 116) acknowledge bothfunctions for their checklists for informational Web sites:“Answering the following questions will help a user deter-mine whether the information on a Web page is comingfrom an authoritative, accurate, and reliable source. . . .The questions can also be used by Web authors as a guideto creating pages that can be recognized as originatingfrom a reliable, trustworthy source.”

The Web site evaluation checklist by Information &

Figure 2. Sample from usability evaluation checklist by Information & Design (1998).



Design (1998; see Figure 2) is an example of heuristicswithout a clear function. On the one hand, the items seemto be meant to identify problems in a Web site—at least,they cannot be summarized into overall judgments aboutthe quality of a site. On the other hand, the checklist formatseems to distract from specific problems, requiring theusers only to specify whether a Web site “always,” “some-times,” or “never” complies with the guidelines.

Apart from the two functions mentioned above, thereis also a third function, which is not very prominent in theliterature on document evaluation. Heuristics may alsohave an idea-generating function. In addition to helpingWeb designers find the problems visitors may have, theheuristics can also present designers with possible designoptions and solutions they were not aware of.

Assumptions about actual useRegrettably, many Web heuristics remain vague when itcomes to the way Web designers are supposed to use themin practice. Some heuristics may be designed for internalrepresentation: They are to be internalized by the Webdesigner before application. The number of heuristic itemsmust be small to be remembered, or, if larger, the heuristicsmust be well structured.

Given the number of different items contained in them,most of the heuristics we encountered seem to be designedfor external representation: They are to be used as a check-list during the evaluation activities. Particularly with thelarge checklists, applying all the heuristic items to the entirecontents of a Web site seems almost unfeasible. For in-stance, based on Nielsen’s 10 usability heuristics, Pierotti(1995) compiled an elaborate checklist for evaluating sys-tems or interfaces, containing 296 items—for example,“Does every display begin with a title or header that de-scribes screen contents?” and “Is there a consistent icondesign scheme and stylistic treatment across the system?”Each item requires an evaluator to check the entire system.The complexity of such evaluation tasks—moving throughan entire site and moving through an entire checklist—canbe expected to have an adverse effect on the usability ofheuristics in practice.

CHARACTERIZING THE RANGE OF HEURISTICSIn this section we show how to apply the frameworkpresented above and demonstrate what it can reveal aboutWeb design heuristics. At the same time, we present threeexisting heuristics that together give a good impression ofthe range of heuristics that we found. We selected thesethree on the basis of their coverage (general or specific,and if specific, the site characteristics they focus on) andtheir presentation format (see Table 1).

Example 1: IBM guidelines fordesigning easy-to-use Web sitesSome heuristics or guidelines try to cover both the com-plete Web design process and the resulting prototype orproduct. The IBM Web design guidelines are an example ofthis kind (IBM 1999). In the left-hand frame in Figure 3, theauthors of the IBM guidelines divide the design processinto four stages: Planning, Design, Production, and Main-tenance. For each of those stages, information about sev-eral sub-processes is given. Overall, the IBM heuristic con-tains 104 items, each of which has the form of aninstruction accompanied by explanations of why and how.

Information covered The IBM guidelines claim to “pro-vide information about user-centered design methods andrecommendations on a variety of issues that challenge Webdevelopers.” They are general in the sense that the guide-lines are organized according to a process approach, withproduct guidelines to be found within the Design section ofthe process model. Designers are supposed to apply theguidelines in the order presented, since some guidelinesbuild on results of previous guidelines. The guidelines aremeant for “your organization’s Web site,” without specificattention to the type of Web site the organization might bedeveloping.

At first sight, the IBM guidelines appear to be compre-hensive. The Web design process instructions draw fromthe methodology of software engineering and project man-agement, and follow the process models from that domain.At a more detailed level, the selection of items within theguidelines is arbitrary at times. For example, the instruc-

TABLE 1: HEURISTICS USED TO DEMONSTRATE THE APPLICATION OF THE FRAMEWORK

Heuristics Coverage Presentation format

IBM (1999) General (process and product) Instructions

Instone (1997b) Specific: User interface design Principles

Alexander and Tate (1999b) Specific: Information quality Questions



tions about text within the Design section contain thefollowing items:

r Create effective headings and place important infor-mation first.

r Keep links separate from narrative text blocks.r Design for default browser fonts.r Make paragraph text flush left.r Test for readability.r Provide a means for users to print groups of related

pages.It appears that the IBM text guidelines focus on text

display on screens only, and not on the many other aspectsof text use, such as style or comprehensibility.

Validity The introductory note in the IBM guidelines isquite clear about their foundation: “These guidelines stemprimarily from our expertise in process and methodology.. . . They also borrow from our experience exploring newinterface designs and developing one of the most usedWeb sites in the world.” The guidelines’ authors claim thattheir information can be particularly valuable for novicesand intermediate level developers. Yet the guidelines showlarge variations in the expertise required to apply them.Some are mechanistic and do not require any expertise atall when applied in a design or review process, such as theguideline “Make paragraph text flush left.” Others, how-ever, assume design expertise, intuition, and creativity that

is not carried over in the guideline—for example, “Designin a style that will appeal to your audience’s tastes.” Al-though the advice makes perfect sense, it does not give adesigner much of a clue about what to do or which designoption to choose.

Presentation format The IBM guidelines are presentedas a combination of instructions and explanations (seeFigure 3), structured by process stage. The instructions inthe headings are procedural, such as “Create an audience/user profile.” The explanatory text that follows elaborateson why one should perform the suggested activity and howto perform it. Even when particular items are product-oriented (such as those on Visual Layout and Elements),they sometimes contain process recommendations, such asthe advice to test the visual design with users. The instruc-tions and explanations have an open format. The formula-tion of most items within the heuristics is high-level ratherthan low-level, with low-level items particularly to befound in the product-oriented items. Those describe prod-uct details rather than a problem observed.

Use The IBM guidelines contain no information abouthow they are supposed to be used. They are meant to beapplied throughout the design process. Their functionseems to be both idea-generating and troubleshooting.They help to generate ideas about what activities to plan

Figure 3. Sample screen of the IBM guidelines (IBM 1999).



and conduct in the design process, and thus can also be anevaluation tool for a post-design process evaluation. Theproduct-oriented guidelines within the Design section ofthe guidelines might be most useful when reviewing designproposals or prototypes. The IBM guidelines are lengthyand at some points very detailed. A printout contains over40 pages. Thus, we can assume that the heuristics will beused “externally,” as a handbook that guides designersthrough the design and development process.

Our evaluation Most of the IBM process guidelines willwork well to make novice and intermediate developersaware of sensible activities in the design process, activitiesthat indeed will help them to create user-centered Websites and pages. We doubt whether the product-orientedguidelines will work well because their quality varies con-siderably, both in terms of validity and in terms of noveltyvalue for designers. Therefore, the guidelines might disap-point Web designers who are seeking clear product guide-lines. We like the combination of concise instructions andadequate explanations, and expect that it will work well inpractice, particularly for the process-supporting items inthe heuristics. Because of the process focus, the length ofthe guidelines will not be prohibitive for those who use theguidelines during an extended design process. For Webdesigners who are looking for a handbook on the Webdesign process (rather than on the product), the IBM guide-lines are a good choice.

A number of heuristics that came up in our searchresemble the IBM guidelines in coverage or in presentationformat. Just like IBM, corporations like Ameritech (1996),Sun (1995), and many smaller players on the market havepublished their corporate Web style guides. Some of theseare very product-oriented, in fact describing the corporategraphic house style for sites; others are more general. Theprocess approach underlying the IBM guidelines is some-what similar to the Yale guide for designing Web sites(Lynch and Horton 1997) and the Hackos and Stevenschecklists for publishing online information for the Web(Hackos and Stevens 1997).

Example 2: Instone’s usabilityheuristics for the WebThe field of human-computer interaction design (HCI) fo-cuses on the design of highly “usable” systems. Usabilitycan be broadly defined as “ease of use,” including suchmeasurable attributes as learnability, speed of user taskperformance, user error rates, and subjective user satisfac-tion. One of the most prolific writers about interface us-ability is Jakob Nielsen (see, for example, 1993, 1999,2000). He summarized findings of interface usability stud-ies into 10 principles that he says are the crucial determin-ers of usability (Nielsen 1994). Keith Instone translated

Nielsen’s 10 principles into Web heuristics (Instone 1997b).Figure 4 shows a sample page with Nielsen’s principles andInstone’s adaptation of them for the Web. The screen cap-ture demonstrates how Instone builds on the content andformat of Nielsen’s guidelines (in bold on the screen).

Information covered Instone’s heuristics on the surfaceappear to be general, but in fact are specific. With regard tothe genre, they certainly are general; they are even appli-cable to a much broader range of user interfaces than Websites alone. With regard to the site features, however, thereare limitations in scope, although Instone does not explic-itly acknowledge this fact. For instance, the heuristicshardly focus on the selection of content for Web sites, orthe comprehensibility of text. They address usability ofinterfaces in a narrow sense: The usefulness of the site forusers is not a topic for Instone. His remark that the “over-riding theme for applying those heuristics to the Web is touse links effectively” demonstrates his shallow view ofusability.

While Nielsen’s underlying 10 principles are based ona factor analysis of usability problems in software interfaces(Nielsen 1994), Instone’s adaptation for the Web reducessome of the principles in an arbitrary way. For example,one of Nielsen’s principles is about the match between thesystem and the real world (see Figure 4). In his annotationof the principle, Nielsen translates “match” into three in-structions:

r The system should speak the users’ language.r It should follow real-world conventions.r It should make information appear in a natural and

logical order.Instone’s adaptation of this principle for the Web

reads: “On the Web, you have to be aware that users willprobably be coming from diverse backgrounds, so figuringout their ‘language’ can be a challenge.” One can wonderwhat exactly is the nature of the advice given here. Wehave the feeling that Nielsen’s second and third instructionare not addressed, and that the first instruction is addressedwith a different interpretation. And what exactly are Webdesigners expected to do or consider when confrontedwith Instone’s advice?

Validity Nielsen’s basis for his 10 principles is made clear(Nielsen 1994), but it remains unproven whether thoseproblems also are valid for Web sites. Instone does notexplain the basis on which he adapted the principles forWeb sites. We assume that his adaptations are practice-based, rather than theory or research-based. The ten prin-ciples leave designers and reviewers much room for theirown interpretation. One of the principles, for example, is:“Error prevention. Even better than good error messages isa careful design which prevents a problem from occurring



in the first place.” Such an instruction does not work well asan aid to avoid problems. What must Web designers do toprevent problems from occurring? Instone’s elaborationsfor the Web vary with regard to the room they leave forinterpretation by the evaluator. Sometimes he points outproblem areas in general—for instance, when he remarks“Probably the two most important things that users need toknow at your site are ‘Where am I? ’ and ‘Where can I gonext?’” At other times, he gives detailed instructions, suchas “Consider using GET instead of POST on your forms.”

As in all heuristics, the novelty value depends on theexperience the user brings to the task. Instone himself isnot too optimistic about the novelty of his heuristics: “Ifyou look at these heuristics and think ‘these are obvious,’then good” (Instone 1997a). He views the heuristics as away for reviewers to formally consider each rule to makesure that they are not forgetting anything. Yet the fact thatthey transfer knowledge from one specific field (designingusable interfaces) to another field (designing usable Websites and pages) might make them novel for some design-ers.

Presentation format The 10 items in the Instone heu-ristics consist of the issues and their explanation (as for-mulated by Nielsen) with Web-focused additions and elab-orations by Instone. They are exclusively product-oriented.Nielsen’s 10 principles are by their nature very open—for

example, “Visibility of system status.” His explanationsare formulated as system requirements (“The systemshould . . .”) or required system effects on users (“Usersshould not have to wonder . . .”). Instone’s additions areboth high level and low level: descriptions of problemareas, instructions, and sometimes very detailed sugges-tions for solutions. His focus on solutions rather than prob-lems, in our view, restricts the usefulness of the heuristics.The heuristics contain 10 items that are numbered but notordered in any way. They can be applied in a random orderwhen designing or reviewing a site.

Use The Instone heuristics are meant to support review-ing a design prototype (heuristic evaluation). The formu-lation of the items as issues could support the use of theheuristics in the design stage, whereas the explanation ininstructive text points to usage as a problem detection anddiagnosis tool. Given the limited number of items (10)within the heuristics, one may expect Web designers anddevelopers to internalize them. The lack of coherence andorder between the items, however, reduces the chance thatdesigners or reviewers can “read between the lines” anddetect problems that are in line with the heuristics but notexplicitly covered by the items.

Instone describes how he expects the heuristics to beused. In his view, a few experts or people with just an hourof training are to be given information about the intended

Figure 4. Sample screen of Instone’s usability heuristics for the Web (Instone 1997b).



audience and purpose of the site or the prototype to beevaluated. Additional scenarios or a site map are to beoffered to help the evaluators become acquainted with thesite. “Then the evaluators make a few passes through yourdesign and after a short while, they can start listing theusability problems they found” (Instone 1997a). Instonestresses that each individual evaluation is “an opinion andshould be treated as such.” Hence, his next step is com-bining 3–5 evaluations and compiling them in a list ofproblems that evaluators then rate for severity. This listhelps designers to prioritize what to focus on during revi-sion.

Our evaluation Our framework helped us to see thatNielsen’s 10 principles are not straightforwardly applicableto Web design. For Web designers, the information cov-ered by the 10 underlying interface usability principles isspecific and hence only appropriate for particular contextsof use. Furthermore, we think Instone’s adaptation of theprinciples is arbitrary and reduces the value of the 10principles as heuristics rather than enhances them. Weregret that one of the very few heuristics that is based onmore than practitioners’ experiences and that provides anexplanation of how it should be used is so unsatisfactory inmany other regards.

Instone’s list with 10 principles is much shorter thanthe IBM guidelines in example 1, a fact that increases thechance that evaluators and designers will internalize theprinciples and indeed apply them. It reduces the task ofevaluating a Web site to checking a Web site against a listof 10 specific requirements. Such a reduction could possi-bly lead to the detection of usability problems in a narrowsense (which is the purpose of the 10 principles), but itdoes not say much about the overall quality of the site.Although the list of principles is designed as an evaluationtool, it might actually work better in the planning stages ofa design process to focus a designer’s attention.

Most of the heuristics that we found through our In-ternet search seem to come from the same field as theInstone/Nielsen heuristics: the field of user interface designand usability studies (see, for example, Story 1999;Tognazzini 1998). They often include a limited number ofitems, a fact that makes them arbitrary or simplistic in lightof the many factors that can influence the usability of aWeb site.

Example 3: The Alexander and Tatechecklists for information qualityOur third example comes from a discipline that has dealtwith assessing value and distributing information sourceslong before the Web came into existence—that is, the fieldof Information or Library Sciences. Alexander and Tate(1999b) have recently published a book on how to evalu-

ate and create information quality on the Web. Their bookconcludes with two appendixes containing checklists in-tended to help the user determine whether the informationon a Web page is derived from an authoritative, accurate,and reliable source. Similar but less extensive heuristics canbe found on their Web site (1999a; see Figure 5). We usedthe heuristics in the book for our analysis.

Information covered The Alexander and Tate heuris-tics (1999b) contain a mix of general and specific items.The authors do not claim to cover all aspects of Web sitesbut support assessing the quality of the information on thepage. Five checklists help users and designers to assessinformation quality using “traditional” criteria, such as re-liability and accuracy. The set of checklists is comple-mented with three checklists on navigation, interaction/transaction, and non-text features. For the latter checklists,the items are certainly not exhaustive for the features cov-ered, and the relation to information quality is unclear. InAlexander and Tate’s Appendix A, the checklists areadapted to a number of main genres of Web sites; thechecklists in their Appendix B are not genre-specific butprimarily feature-specific, focusing on information quality.

Validity The issue at stake in the Alexander and Tateheuristics is not the information quality of the site but ratherthe degree to which information quality can be assessedthrough content and page features such as author informa-tion, update frequency, or contact information. The ques-tions in the Alexander and Tate heuristics often have ayes/no answer format, which makes them seem mechanis-tic—for example, “Are the qualifications of the organiza-tion, company or person responsible for the content of thesite indicated?” Some questions leave more room for inter-pretation by the user of the heuristics: “Is the point of viewof the individual or organization responsible for providingthe information evident?” Different reviewers might wellanswer that question differently. Since the heuristicspresent information from a field that is not typically asso-ciated with Web design, the heuristics might well containnovel information for evaluators and designers.

Presentation format The checklists consist of long listsof yes/no questions. Explanations are found throughoutthe book preceding the checklists. The questions are orga-nized under headings such as “The business checklist” (inAlexander and Tate’s Appendix A) or “Objectivity” (in theirAppendix B). The formulation mostly refers to specificproduct characteristics, such as the presence of a logo or asite map. Hence, the issues are low-level issues, rather thanhigh-level ones.

Use Alexander and Tate indicate that their heuristics are



primarily meant for evaluation purposes: “Answering thefollowing questions will help a user determine whether theinformation on a Web page is coming from an authorita-tive, accurate and reliable source.” They can also be usedfor planning purposes: “. . . as a guide to creating pagesthat can be recognized as originating from a reliable trust-worthy source.” Given the number of items and the formu-lation in detailed product characteristics, the heuristics sup-port the evaluation stage better than the planning stage.The structure of the heuristics and the subdivision alonglines of genres helps designers internalize the long lists ofquestions. Alexander and Tate suggest using the checklistfor verification: “The greater the number of ‘yes’ answers,the greater the likelihood that the quality of the informationon the page can be determined.” They are not clear aboutthe number of “yes” answers required to benchmark a pageas containing “high quality” information.

Our evaluation The five Alexander and Tate checklistson information quality criteria will be useful aids for Webdesigners and evaluators. They focus on an issue that mightwell be very new to Web designers who do not have anInformation Science background. They draw attention toproduct and content characteristics that can serve as asignal for information quality on Web pages. The focus onthe signals rather than on the content is at the same time alimitation. The issue of information quality from the pointof view of users (for example, comprehensibility, applica-

bility) is underdeveloped. The adaptation of the checkliststo various types of Web pages makes sense: the purposeand functions of the site indeed influence the kinds ofsignals to be expected on the page.

We are less convinced of the value of Alexander andTate’s checklists on interaction and transaction features,navigation, and non-text features. If the goal is to evaluatethose features, better heuristics are available. Probably theauthors have included checklists on those features becausethey assume that they have a special influence on informa-tion quality. However, they have not succeeded in formu-lating checklist questions on these features that help toidentify signals of that influence.

The term checklist and the use of the yes/no answerformat suggests use in an evaluation stage rather than in aplanning stage. Using the checklist for evaluation can be-come very complicated since evaluators have to movethrough the entire site and through the entire checklist atthe same time. However, the clear structure of the check-lists enables evaluators to “read between the lines.”

Many relevant sources on evaluating the quality ofWeb-distributed information can be found through Alex-ander and Tate’s Web site (1999a). Heuristics on assessingthe quality of Web information often come from universitylibraries that support students who want to use informationfor research papers—for example, Grassian (1998) of theUniversity of California Los Angeles and Ormondroyd,Engle, and Cosgrave (1999) of Cornell University. Those

Figure 5. Sample screen of the Alexander and Tate heuristics (1999a).



heuristics have a less broad and thorough view of informa-tion quality than the Alexander and Tate heuristics.

TOWARDS A RESEARCH AGENDAWhile we were writing this article, participants in an onlinediscussion list on usability testing were discussing the valueof heuristic evaluation as opposed to testing with users.One practitioner recommended including heuristic evalu-ation in all usability testing projects prior to involvingrepresentatives of the target groups. In reply, another par-ticipant suggested that designers skip heuristic evaluationaltogether when user-focused testing is planned, since“. . . the first five users uncover 70% of the major usabilityproblems and it’s rarely necessary to test more than eight. . . . it’s proven true in hundreds of tests we’ve conducted.”The argument between the list participants simply cannotbe resolved, since at the moment, we lack a sound foun-dation for discussing the value of heuristic evaluation. Theframework we present in this article may contribute invarious ways to discovering the answer to this question ofthe value of heuristic evaluation.

Framework as an aid for reflectionOur framework can be used as an aid to reflect on the prosand cons of existing heuristics, both in general and inspecific Web design processes. It is not clear what tech-niques or guidelines are meant exactly when the termheuristic evaluation or heuristics is used. Given the diver-sity of heuristics we found and the inconsistency in names,we should first distinguish a number of approaches andcharacterize them. We have presented our framework forcharacterizing heuristics as a first step in this direction. Thecategories in the framework we present—coverage, valid-ity, presentation format, and use—reflect the characteristicsthat we think are important. Using the framework is notmerely an academic inventory exercise: Practitioners needto reflect on the tools they are using and the kind ofcontribution they expect their tools to make in their designpractice. If they do not carefully consider the nature andaffordances of their tools, they will end up trying to drivescrews in the wall with a hammer.

Framework revealing success factorsThe framework has given us some ideas (albeit speculativeones) about critical success factors for heuristics. Withregard to the categories of coverage and validity, our mainconclusion is a rather obvious one. Heuristics should notconsist of arbitrary items, and the users of heuristics shouldbe adequately informed about the rationale behind theheuristics. As obvious as these requirements may be, noneof the existing heuristics fully meets them.

With regard to the presentation and use of heuristics,three requirements seem to be very important.

r Heuristics should enable Web designers to “read be-tween the lines”—that is, to fill in the gaps betweenheuristic items and to see the intentions behindthem. A meaningful structure and an adequate mixof high-level and low-level items may be very im-portant in this respect.

r Heuristics should inform users about the backgroundand rationale of the items contained, making it pos-sible for users to evaluate the applicability of theheuristics for a specific site or design process, and,again, to see the intentions behind them. The linkbetween heuristic items and supporting evidencemust therefore be emphasized.

r The nature and formulation of heuristics should betailored to the ways heuristics are to be used in theWeb design process.

In all three respects, serious flaws are apparent in many ofthe existing heuristics.

Framework to point out research issuesFinally, the use of the framework has made clear how littlewe know as yet about the use and the benefits of heuristics.In research into the effectiveness of heuristics, two mainissues need to be addressed.

1. We need to know more about the ways heuristicsare actually used in design and evaluation practices inrelation to their presentation format. That knowledge isindispensable for developing sound heuristics, as muchas observing people who are reading and using manualshas been and still is important for technical communica-tors, or thinking-aloud studies of people using softwarehave been and still are important for interface designers.

2. But practitioners’ reports about their use of heu-ristics or about the results of using heuristics (like Leviand Conrad 1996) is not enough of a basis to determinewhat makes heuristics effective. The second issue on ourresearch agenda is that we need to know more aboutwhat the heuristics actually did for the Web designers. Atthe moment we have lots of practitioners advocatingheuristic evaluation, but very few solid reports on theyield of heuristics and heuristic evaluation. The few stud-ies we have (for example, Nielsen 1994; Desurvire 1994)compare heuristic evaluation to user-focused testing, butthere is a place for both evaluation approaches in thedesign process. It is no use to create an artificial opposi-tion between methods and to study them as if we needto declare one better than the other.

There are many more relevant questions to be an-swered. The studies of Nielsen and Desurvire indicate thatthe various reviewers using the heuristics did not agreemuch about the problems they saw, and did see problemsthat were not detected by the users. What kinds of prob-lems can be predicted or detected by users of heuristics,



and how do they compare (in a qualitative, not in a quan-titative sense) to the results of unguided evaluation or othermethods of testing? What is the relation between the actualheuristics used and their characteristics on one hand, andtheir yield on the other hand? What is the role of priorknowledge and experience or task perception of the userof the heuristics? And (here we come to what we think isthe ultimate question about the effectiveness of heuristics)does using heuristics or using the findings of heuristicevaluation indeed result in better Web sites or better pages?

A starting point for practitioners and researchersOur Web search located 384 heuristics, and by the time thisarticle appears in print, this number will undoubtedly havegrown. Web designers and producers find it important todevelop heuristics and to present them; their colleaguesappreciate the sharing of experiences and find heuristicsvaluable tools for their design practice. Despite the largenumber of heuristics, however, the number of studies ofwhat heuristics do is very small. We don’t know whatmakes heuristics work as tools in the design process. Thereis still much practical and research work to do before wecan use those tools to their full extent, but we are fasci-nated by their potential. We hope this article and thespecial issue it is part of will work as a starting point forboth practitioners and researchers to further develop thepromising set of instruments that heuristics can be for Webdesigners. TC

REFERENCESAlexander, J. E., and M. A. Tate. 1999a. “Evaluating Web

resources.” http://www2.widener.edu/Wolfgram-Memorial-Library/webeval.htm

Alexander, J. E., and M. A. Tate. 1999b. Web wisdom. How toevaluate information quality on the Web. Mahwah, NJ:Lawrence Erlbaum.

Ameritech. 1996. “Ameritech Web page user interface standardsand design guidelines.” http://www.ameritech.com/corporate/testtown/library/standard/web_guidelines/index.html]

Byard, V. 1991. “Ideology in writing instruction: Reconsideringinvention heuristics.” Paper presented at the 42nd AnnualMeeting of the Conference on College Composition andCommunication, Boston, MA.

CAST (Center for Applied Special Technology). 1999. “Bobby.”http://www.cast.org/bobby

de Jong, M., and P. J. Schellens. 1997. “Reader-focused textevaluation: An overview of goals and methods.” Journal ofbusiness and technical communication 11:402–432.

Desurvire, H. W. 1994. “Faster, cheaper!! Are usability inspectionmethods as effective as empirical testing?” In Usabilityinspection methods, J. Nielsen and R.L. Mack, eds. New York,NY: John Wiley & Sons, pp. 173–202.

Evans, M. B. 2000. “Challenges in developing research-basedWeb design guidelines.” IEEE transactions on professionalcommunication 43 (in press).

Gerhardt-Powals, J. 1996. “Cognitive engineering principles forenhancing human-computer performance.” International journalof human-computer interaction 8, no. 2:189–211.

Grassian, E. 1998. “Thinking critically about World Wide Webresources. ” http://www.library.ucla.edu/libraries/college/instruct/web/critical.htm

Hackos, J. T., and D. M. Stevens. 1997. Standards for onlinecommunication. New York, NY: John Wiley & Sons. “HumanFactors International.” 1999. http://www.humanfactors.com/home

IBM. 1999. “IBM ease of use. The World Wide Web.”http://www.ibm.com/ibm/easy/design/lower/f060100.html

Information & Design. 1998. “Usability evaluation checklist forWeb sites.” http://www.infodesign.com.au

Instone, K. 1997a. “Site usability evaluation.” http://webreview.com/pub/97/10/10/usability/index.html

Instone, K. 1997b. “Usability heuristics for the Web.”http://webreview.com/wr/pub/97/10/10/usability

Levi, M. D., and F. G. Conrad. 1996. “A heuristic evaluation of aWorld Wide Web prototype.” Interactions magazine 3, no. 4:51–61.

Lynch, P. J. and S. Horton. 1997. “Yale C/AIM WWW stylemanual.” http://informed.yale.edu/caim/manual/contents.html

Nielsen, J. 1993. Usability engineering. Boston, MA: AcademicPress.

Nielsen, J. 1994. Heuristic evaluation. In Usability inspectionmethods, J. Nielsen and R. L. Mack, eds. New York, NY: JohnWiley & Sons, pp. 25–62.

Nielsen, J. 1999. “Useit.com: Jakob Nielsen’s Website.” http://www.useit.com

Nielsen, J. 2000. Designing Web usability: The practice ofsimplicity. Indianapolis, IN: New Riders.



NIST (National Institute of Standards and Technology). 1999.“Webmetrics.” http://zing.ncsl.nist.gov/webmet

Oliver, K. M., G. L. Wilkinson, and L. T. Bennett. 1997.“Evaluating the quality of Internet Information sources.”http:itech1.coe.uga.edu/faculty/gwilkinson/aace97.html

Ormondroyd, J., M. Engle, and T. Cosgrave. 1999. “How tocritically analyze information resources.” http://www.library.cornell.edu/okuref/research/skill26.htm

Pierotti, D. 1995. “Heuristic evaluation—A system checklist.[Usability Analysis & Design, Xerox Corporation.]” http://stc.org/pics/usability/topics/he-checklist.html

Scholtz, J., S. Laskowski, and L. Downey. 1998. “Developingusability tools and techniques for designing and testing Websites.” Proceedings of the 4th Conference on Human Factorsand the Web. http://www.research.att.com/conf/hfweb/proceedings/scholtz/index.html

Story, D. 1999. “Usability checklist for site developers.” http://webreview.com/pub/1999/10/15/usability/index.html

Sun Microsystems. 1996. “Sun on the Net. Guide to Web Style.”http://www.sun.com/styleguide/index.html

Tognazzini, B. 1998. “Ask Tog.” http://www.asktog.com/basics/firstPrinciples.html

Tohsaku, C. 1997. “Guidelines for Web design. [University ofCalifornia San Diego.]” http://cybermed.ucsd.edu/Web_design/process.html

van der Geest, T. (in press). Web site design is communicationdesign.

W3C (World Wide Web Consortium). 1999. “Web contentaccessibility guidelines 1.0. W3C recommendation 5-May-1999.” http://www.w3.org/tr/wai-webcontent/wai-pageauth.html

Wright, P. 1985. “Editing: Policies and processes.” In Designingusable texts, T. M. Duffy and R. Waller, eds. Orlando, FL:Academic Press, pp. 63–96.

W-SDA. 1997. “Web-site design audit. [State University of NewYork at Buffalo.]” http://www.eng.buffalo.edu/;ramam_m/au_fr.html



MENNO DE JONG is an assistant professor of communicationstudies at the University of Twente, Enschede, the Netherlands. Hisresearch interests concern the use and methodology of applied re-search to optimize communication. He has published about formativetext evaluation, usability, and document design. His PhD dissertation(1998) deals with the value of the plus-minus method for pretestingpublic information brochures. Contact information: [email protected]

THEA VAN DER GEEST works with the Department ofCommunication Studies at the University of Twente, the

Netherlands. She teaches courses on new media and commu-nication technology, the design of Web sites, project man-agement, and written communication. Her research focuseson the process of effective document design for newmedia. Her publications include books and articles on thenew writing environment, the writing process, and theWorld Wide Web. She is the New Media editor for the Docu-ment design journal. Her book on Web site design as com-munication design will appear shortly from John BenjaminsPublishers. Contact information: [email protected]

Presentation format of the heuristics

Structure Randomly ordered 7 Meaningfully structured

Formulation of items InstructionsQuestionsRequirements

Type of answers Open 7 Closed

Level of items High-levelLow-level

Use of the heuristics

Phase in the design process Planning-orientedEvaluation-oriented

Focus of support Process-orientedProduct-oriented

Function in the design process TroubleshootingVerifyingIdea-generating

Assumptions about actual use Internal representationExternal representation

APPENDIX 1: A FRAMEWORK FOR CHARACTERIZATING HEURISTICS

Information covered by the heuristics

Specificity GeneralGenre-specificFeature-specific

Exhaustiveness Exhaustive 7 Arbitrary

Validity of the heuristics

Foundations StandardsTheoryResearchPractitioners

Novelty value High 7 Low

Room for interpretation Mechanistic 7 Expert-mediated

Validation research Available 7 Unavailable



characterizing web heuristics - simon fraser...

Documents