Disability and Health Journal 7 (2014) 26e35

Review Article

Measurement properties of instruments that assess inclusive accessto fitness and recreational sports centers: A systematic review

Allyson M. Calder, B.H.Sc. (Physiotherapy), PG.Cert.CT.*, andHilda F. Mulligan, B.Sc. (Physiotherapy), M.H.Sc., Ph.D.

Centre for Health, Activity, and Rehabilitation Research, School of Physiotherapy, University of Otago, New Zealand

www.disabilityandhealthjnl.com

Abstract

Background: Exercise is necessary for overall health and well-being for all individuals. For people with disabilities, fitness andrecreational sports centers are reported to be generally inaccessible and not user friendly.

Objective: This review study aimed to identify instruments that assess access to fitness and recreational sports centers and to appraisethe identified instruments’ qualitative and quantitative attributes.

Methods: We systematically searched databases (AMED, CINAHL, EMBASE, MEDLINE, SCOPUS, SPORTDiscus and Web ofScience for the years spanning 1950 to April 2012) and web-based search engines (Google and Google Scholar) to identify instruments,published in English that objectively assess the accessibility of fitness and recreational sports centers. Identified instruments were criticallyappraised using the qualitative attributes QAPAQ Checklist Part I and the COSMIN checklist for measurement properties.

Results: Seven instruments were included in this review: ADA Accessibility Guidelines Checklist for Buildings and Facilities(ADAAG); ADA Checklist for Readily Achievable Barrier Removal; Accessibility Instruments Measuring Fitness and Recreation Environ-ments (AIMFREE); Community Health Environment Checklist e Mobility (CHEC-M); Removing Barriers to Health Clubs and FitnessFacilities (RBHCFF); Health Empowerment Zone Environmental Tool Shortened Environmental Checklist: Fitness Centre Survey(HEZEAT-FCS); Community Health Environment Checklist e Exercise Facilities (CHEC-Fit). Only the AIMFREE and CHEC-M haveaspects of measurement properties evaluated.

Conclusion: We recommend that instrument developers consider conducting full psychometric assessment of their instruments usingadequate sample sizes. We also recommend they consider scoring methods and respondent burden to provide scientifically robustinstruments that are easy to administer. � 2014 Elsevier Inc. All rights reserved.

Keywords: Fitness centers; Architectural accessibility; Environment design; Disabled persons

People with disabilities (PwD) are amongst the mostphysically inactive groups in society.1e5 As such this popu-lation group is at high risk of developing secondary condi-tions which may impact negatively on their physicalfunction and overall well-being.6e8 Participation in physicalactivity through recreation has substantial benefits to mini-mize secondary conditions.7,9 To enable PwD to becomephysically active, indoor fitness and recreational sportscenters (defined by the North American Industry Classifica-tion System (NAICS)10 as, ‘‘establishments primarilyengaged in operating fitness and recreational sports facili-

Conflict of interest and statement of funding: The authors have no

financial or conflicts of interest to declare.

* Corresponding author. School of Physiotherapy Christchurch, Univer-

sity of Otago, PO Box 4345, Christchurch Mail Centre 8140, New Zealand.

Tel.: þ64 3 364 1826.

E-mail address: ally.calder@otago.ac.nz (A.M. Calder).

1936-6574/$ - see front matter � 2014 Elsevier Inc. All rights reserved.

http://dx.doi.org/10.1016/j.dhjo.2013.06.003

ties featuring exercise and other active physical fitnessconditioning or recreational sports activities’’) need to beeasily accessible and provide inclusive, safe and supportiveenvironments.11 However, international evidence suggeststhat a large proportion of PwD do not use fitness and recrea-tional sports centers.7 PwD repeatedly report poor ‘‘accessi-bility’’ and ‘‘usability’’ of these environments for them.12e17

The term ‘‘accessibility’’ is an objective descriptor basedaround Iwarsson and Stahl’s18 concept of ‘‘personeenvir-onment fit.’’ This entails a dynamic relationship betweena person’s functional capacity and a particular physicalenvironment. ‘‘Usability’’ takes the notion of accessibilityone-step further. It describes the perception of how an envi-ronment restricts or supports an individual.18 For example,an individual with a physical disability might be able toaccess and move around the building, but if they cannotmake use of the equipment safely, then the fitness facilitywould not be usable to such an individual. Studies by

27A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

Rimmer et al5 and Rolfe et al19 have explored barriers andfacilitators of community fitness facilities by PwD. Theseauthors have highlighted the importance of a structuredmeans of measuring access to fitness and recreational sportscenters for PwD.

Over the past twenty years the terms ‘‘accessibility’’ and‘‘usability’’ have increasingly gained attention (at least ina theoretical sense), as policies and legislation havepromoted equal opportunities for PwD to participate in aninclusive society.20,21 In many countries, building regula-tions (codes and standards) require that there be accessibilityfor all.21e23 In the United States of America (USA), theAmerican Disabilities Act (ADA) 1991 is the law that under-pins accessibility for all. The ADA prohibits discriminationagainst PwD for accessibility in all areas of life, includingrecreational exercise.24e27 Titles II and III of the documentspecifically refer to the provision of equal access to publicfacilities and set out the minimum legal standards forachieving accessible environments. Most community fitnessand recreational sports centers fall under the umbrella ofpublic facilities and therefore are required by USA law toprovide equal access for PwD.1,7 Recent evaluation studiesabout the degree to which fitness and recreational sportscenters in the USA comply with the ADA built environmentdomains (e.g. parking, entrances, restroom, access to equip-ment) for people with mobility impairments have, however,found no facility to be 100% compliant in any do-main.1,21,28e31 Potential reasons include:

i. The building regulation legislation are minimalstandards of compliance23,32,33 which likely suitsthose who are either able bodied or are only mini-mally physically impaired.

ii. The research was undertaken within existing build-ings that were never built to code.1,28e31 UnderADA legislation, owners of existing buildings haveto remove architectural barriers only if they arereadily achievable (i.e. without much difficulty orexpense)34e36 which means they are unlikely everto achieve 100% compliance.

iii. There appears to be a lack of knowledge and aware-ness by building owners, developers, designers, andbuilders about accessibility issues encountered byPwD.36e38

Evaluating fitness and recreational sports centers foraccessibility would allow PwD to become more self-efficacious in their decision-making about which fitnessfacility best meets their individual requirements. For healthprofessionals, such knowledge would be helpful in intro-ducing an individual with disability to a facility that bestmeets their needs in relation to personeenvironment fit.Furthermore, fitness center owners could benefit from usingan instrument that accurately evaluates their facility todetermine accessibility compliance and to identify potentialbarriers to the personeenvironment fit for PwD.

Instruments that assess accessibility of fitness and recre-ation sports centers need to be user friendly and psychomet-rically sound.39,40 The purpose of this paper was to:

i) Systematically search the literature to identifyinstruments that can be used to assess accessibilityof fitness and recreational sports centers

ii) Critically appraise the instruments’ qualitative attri-butes (the instrument’s purpose, setting, construct,justification, target population, format, interpret-ability, feasibility and utility) using the QualityAssessment of Physical Activity Questionnaire(QAPAQ) Checklist Part 141

iii) Critically appraise the instruments’ psychometricmeasurement properties of reliability, validity andresponsiveness using the COsensus-based Stan-dards for the selection of health MeasurementINstruments (COSMIN) checklist42,43

We selected the QAPAQ Checklist Part 1 and the COS-MIN checklist because there is no single checklist availableto appraise qualitative attributes and measurement proper-ties for instruments assessing accessibility of fitness andrecreational sports centers. Terwee et al41 developed theQAPAQ Checklist Part 1 to appraise the qualitative attri-butes of physical activity questionnaires. The checklistgenerates descriptive information about an instrument’squalitative attributes.41 Although the QAPAQ Checklist Part1 has not undergone psychometric evaluation, it has beenused in a number of recently published systematic reviewsthat have investigated measures of health status.44e50

The COSMIN checklist was developed by a group of 43experts in health status measurement across different healthdisciplines using a Delphie approach to classify psycho-metric measurement properties.42,43 The experts developedthe COSMIN checklist originally to evaluate health-relatedpatient-reported outcome (HR-PRO) questionnaires.42

However, the COSMIN developers also state their checklistis suitable for evaluating the psychometric properties ofother instruments.42 Therefore, we considered it suitableto evaluate instruments that measure accessibility.

Methods

Search strategy

We systematically searched databases (AMED (Ovid1985), CINAHL (via Ebsco), EMBASE (Ovid 1947), MED-LINE (Ovid 1950), SCOPUS, SPORTDiscus (via Ebsco),and Web of Science via Web of Knowledge for the yearsspanning 1950 to April 2012) and web-based search engines(Google and Google Scholar) using key words and syno-nyms for: recreation; environment; accessibility; instrumentand evaluation. We eliminated duplicate instruments beforescreening them for eligibility. Included instruments (writtenin the English language) objectively measured accessibilityof fitness and recreational sports centers.

28 A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

Quality appraisal

We conducted a two-stage approach to appraise theincluded instruments. First, two reviewers independentlyappraised each instrument’s qualitative attributes descrip-tively using the QAPAQ Checklist Part 141 to extract theinstruments’ purpose, setting, construct, justification, targetpopulation, setting, format, interpretability and feasibility(ease of use). In addition, we recorded the instruments’utility. Second, two reviewers independently appraised theinstruments’ psychometric measurement properties usingthe standardized COSMIN checklist.42 With this checklist,individual items within each domain are scored using a fourpoint ordinal rating scale: excellent, good, fair and poor.51

The COSMIN authors state the ‘‘worse score counts.’’Therefore the lowest score assigned across all the itemswithin a domain becomes the overall score for thatdomain.51 This rating scale has not yet undergone formalpsychometric evaluation. Although it demonstrates soundface validity (through the international Delphie study42,43),the inter-rater reliability has shown low Kappa coefficients(61% below 0.4 and 6% above 0.7552). Mokkink andcolleagues52 suggest this is due to some of the COSMINitems requiring a subjective judgment from the raters.Therefore to lessen the subjectivity when interpreting theCOSMIN items, the two independent appraisers wererequired to come to a consensus of their rating via discus-sion, as suggested by the COSMIN developers.53

Results

The results yielded a total of 88 instruments, three ofwhich were duplicates, leaving a total of 85 eligible forreview. Most instruments (n 5 81) were identified froma report by Rimmer, Gray-Stanley and Haugen54 witha few located through web-based searches (n5 7) and fromelectronic databases (n 5 2). Our initial review revealedthat 78 instruments did not evaluate the built environmentof fitness and recreational sports centers (Fig. 1), and thus,this study reports findings from reviewing seven instru-ments (Table 1).

The ADA Accessibility Guidelines (ADAAG) Checklistfor Buildings and Facilities instrument24,55 arose directlyfrom titles II and III of the ADA legislation (created bythe USA Government in 1991) to objectively measure builtenvironment accessibility. This instrument forms the foun-dation for four other instruments: the ADA Checklist forReadily Achievable Barrier Removal35; the RemovingBarriers to Health Clubs and Fitness Facilities (RBHCFF):An Abbreviated Accessibility Survey27; the AccessibilityInstruments Measuring Fitness and Recreation Environ-ments (AIMFREE)55e57 and the Health Empowerment ZoneEnvironmental Tool Shortened Environmental Checklist:Fitness Centre Survey (HEZEAT-FCS).59 The ADAAG,the ADAChecklist for Readily Achievable Barrier Removaland the RBHCFF instruments measure accessibility. The

AIMFREE, HEZEAT-FCS, Community Health Environ-ment Checklist eMobility (CHEC-M)55,58 and CommunityHealth Environment Checklist e Exercise Facilities(CHEC-Fit)60 measure accessibility and usability.

Qualitative attributes appraisal

Table 2 displays the QAPAQ Checklist Part 1 resultsacross the seven instruments to detail construct and justifi-cation, target population, format, interpretability, feasibilityand utility. In summary, all of the instruments measureaccessibility of the structural built environment (i.e.a building). They are all discriminative instruments. Fiveof the instruments are suitable for users interested inmeasuring accessibility of the structural built environmentin general, and two of these five instruments (the AIM-FREE and CHEC-Fit) target measurement of accessibilityspecifically for fitness and recreational sports centers.Two other instruments (the ADA Checklist for ReadilyAchievable Barrier Removal and the RBHCFF) are foruse by building owners. The RBHCFF specifically targetsfitness and recreational sports center owners. All the instru-ments are obtainable on-line at no cost, except for the AIM-FREE, which is obtainable at a small cost from thedevelopers. None of the instruments are reported to requiretraining for use. The AIMFREE is reported to take 2e3 h toadminister; the CHEC-M is reported to take around 25 minto administer; none of the other instruments report timerequired. All of the instruments use a dichotomous scoringsystem (yes/no responses), however both the AIMFREEand CHEC-M use weighted scoring algorithms. The AIM-FREE uses a Rasch measurement model to calibrate thecharacteristics of fitness and recreational sports centers ac-cording to a weighted score allowing a spectrum of charac-teristics (for example, physical access to the building versussafe access to the equipment) to be examined togeth-er.4,57,69e73 In contrast, although the CHEC-M hasa weighted scoring system, the developers do not providedetail on its use. The ADAAG and the AIMFREE are theonly instruments that appear to have been used in publishedstudies investigating accessibility of fitness and recreationalsports centers.1,7,28,30,61,62,64e68

Measurement property appraisal

Only two instruments (the AIMFREE55e57 and CHEC-M55,58) reported aspects of reliability and validity. We weretherefore able to appraise these according to the COSMINclassification (Fig. 2). The COSMIN classification defini-tions of reliability and validity are displayed in Table 3.In summary, appraisal of the Rasch measurement modelused for the AIMFREE instrument scored as good. Theinternal consistency (a measure of reliability) of the AIM-FREE instrument scored as good, with the CHEC-Mscoring poor. Both instruments scored fair to poor for othermeasures of reliability (testeretest and inter-rater). Bothinstruments scored as excellent for their content validity.

Fig. 1. Systematic search strategy to identify instruments that assess accessibility of fitness and recreational sports centers.

29A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

None of the other instruments report reliability and validitytherefore, we were unable to classify them with COSMIN.

Discussion

A systematic search of published instruments thatmeasure features of the built environment to assess accessi-bility of fitness and recreational sports centers yielded seven

Table 1

Instruments identified for review

Date instrument

published Instrument

1991 ADA Accessibility Guidelines (ADAAG) Checklist

1995 ADA Checklist for Readily Achievable Barrier Rem

2004 Accessibility Instruments Measuring Fitness and Re

2007 Community Health Environment Checklist e Mobil

2008 Removing Barriers to Health Clubs and Fitness Fac

2009 Health Empowerment Zone Environmental Tool Sho

2010 Community Health Environment Checklist e Exerci

for review. Analysis of the qualitative and quantitative prop-erties of these instruments found that while most were devel-oped for public use with no to minimal training, only twohave been evaluated for their psychometric properties.

Regardless of an instrument’s user (researcher, profes-sional or consumer), the instrument should demonstratesound psychometric qualities that include parameters of

for Buildings and Facilities24,55

oval (version 2.1)35

creation Environments (AIMFREE)55e57

ity (CHEC-M)55,58

ilities (RBHCFF): An Abbreviated Accessibility Survey27

rtened Environmental Checklist: Fitness Centre Survey (HEZEAT-FCS)59

se Facilities (CHEC-Fit)60

Table 2

Qualitative attributes of identified instruments

Instrument Construct & justification Target population Format & interpretability Feasibility (ease of use) Utility

ADA Accessibility

Guidelines Checklist

for Buildings and

Facilities

(ADAAG)24,55

Construct

Accessibility for places of

public accommodation,

commercial and transportation

facilities.

Justification

Surveys minimum standards

for legislative compliance with

American Disabilities Act

(ADA) Titles II & III.

Any person with an

interest in

accessibility of the

structural built

environment.

821 clearly described items across 22 assessment areas:

parking; passenger loading zones; exterior accessible

routes; curb ramps; drinking fountains; telephones;

ramps; stairs; platform lifts; entrances and exits; doors

and gates; building lobbies and corridors; elevators;

rooms and spaces; assembly areas; toilet rooms and

bathrooms; bathtubs and showers; dressing and fitting

rooms; signage; alarms; detectable warnings; automated

teller machines.

Accessibility guidelines for special facilities

(transportation, medical care facilities, libraries,

mercantile facilities, restaurants/public housing) are also

included.

Items are scored by a simple ‘‘yes’’ or ‘‘no’’ system as to

whether they meet the minimum requirement of

accessibility or not. Individual items are not summed for

an overall accessibility rating.

Instrument can be used in

part or in full

depending on the

building being

assessed.

Instrument available

on-line at no cost.24

Administered on hard

copy.

Length of time to

complete survey is not

known.

No training required.

The ADAAG has been used

in research studies within

the United States of

America (USA) to

determine ADA

compliance of fitness and

recreational sports

centers.1,28,30 The ADAAG

has also been applied in

research outside of the

USA e Nigeria61 and the

United Arab Emirates.62

ADA checklist for readily

achievable barrier

removal.35

Construct

To aid identification of barriers

concerning the structural built

environment accessibility.

Justification

To assist building owners to

meet ADA legislation.

Building owners A shortened form of the ADAAG instrument which is based

on the four priorities recommended by Title III of the

ADA: accessible entrance; access to goods and services;

access to restrooms; any other measures regarding access

(drinking fountains and telephones).35

104 items are scored in the same manner as the

ADAAG.35 Helpful solutions to remove barriers are

offered to the instrument user if items are scored ‘‘no.’’

Instrument available on-

line at no cost.35

Administered on hard

copy.

Length of time to

complete survey is not

known.

No training required.

This instrument has been

applied in research of built

environments but not

specifically to fitness and

recreational sports centers.

Accessibility Instruments

Measuring Fitness and

Recreation

Environments

(AIMFREE).55e57

Construct

To evaluate the accessibility

and usability of public fitness

and recreational facilities for

people with mobility

impairments (using

wheelchairs or other assistive

devices) and visual

impairments.

Justification

Developers recognized two

major issues that were not

addressed with the ADAAG.

First, the ADAAG only

addressed minimal standards

for built environment

accessibility and the concept

of usability was not addressed.

Second, fitness facilities

remain inaccessible after two

decades of ADAAG inception.

Any person with an

interest in

accessibility and

usability of fitness

and recreational

facilities.

Covers 6 domains related to accessibility (based on the

ADAAG) and usability: built environment; information;

policies; swimming pools; professional behavior

(attitudes and knowledge).

Six instruments are available: fitness center and

swimming pool accessibility survey; a fitness centre

survey; and a swimming pool survey. Each survey is

available in a consumer and professional version. The

number of items varies depending on the version used.

Clearly described weighted scoring system (based on the

Rasch analysis model). At the end of each domain, the

user is able to subjectively rate the overall accessibility of

that domain on a 1e7 point visual analog scale (VAS).

The subjective component is not included in the overall

accessibility rating.

Each domain can be used independently of each other.

All six instruments are

available on-line at a

small cost.

The instrument can be

administered on hard

copy or entered into an

on-line version using

wireless devices

(laptops, notebooks and

smartphones).36,63 Item

responses entered on-

line can provide the

user with potential

solutions to improve

accessibility.36

2e3 h to complete

depending on the

version used.36

No training required.

The AIMFREE has been used

in research studies in the

USA,7,64,65 Canada,66

Kuwait67 and Singapore68

to determine accessibility

and usability of fitness and

recreational facilities.

(Continued)

30

A.M

.Calder

andH.F.

Mullig

an/D

isability

andHealth

Journal7(2014)26e35

Table 2

Continued

Instrument Construct & justification Target population Format & interpretability Feasibility (ease of use) Utility

Community Health

Environment Checklist

e Mobility (CHEC-

M).55,58

Construct

To determine if a building or

facilities features are

ecologically valid.55,58

Justification

The authors believe that one’s

ability to participate in society

is influenced by how people

are able to use their

environment.60

Any person with an

interest in

accessibility of the

structural built

environment.58,60

Particularly

applicable to

individuals with

mobility

impairments.55,58

Accessibility and usability constructs are assessed over 65

items across five domains: building entry using

accessible pathways; using the building; restrooms;

amenities; usability and area of rescue assistance.

A weighted scoring system developed by the authors

where individuals with mobility impairment ranked each

item in order of most to least importance for accessibility.

A mean ranking score was calculated for each item.

Weighted scores from each domain were summed to

create a total accessibility score. How to apply this

weighted scoring system in practice is not discussed by

the developers.

Instrument available on-

line at no cost.60

Administered on hard

copy.

CHEC-M can be

completed in a mean

time of 25 min.

No training required.

CHEC-M to date has not been

applied in any research

studies.

Removing Barriers to

Health Clubs and

Fitness Facilities: An

Abbreviated

Accessibility Survey

(RBHCFF).27

Construct & justification

Developed specifically for

fitness and recreational sports

center owners to determine if

their facility was accessible for

people with disabilities (PwD).

Building owners especifically fitness

and recreational

sports center

owners.

Established from the ADAAG and research studies

investigating accessibility barriers in fitness

facilities,1,27,31 it is an abbreviated accessibility survey

consisting of 125 items across ten sections: customer

service; parking; reception/waiting area; paths and

entrances; signage; elevators; locker rooms; exercise

equipment; pool areas; and emergency procedures.

Scoring is the same as the ADAAG and ADA readily

achievable barrier removal instruments.

Instrument available on-

line at no cost.27

Administered on hard

copy.

Length of time to

complete survey is not

known.

No training required.

RBHCFF to date has not been

applied in any research

studies.

Health Empowerment

Zone Environmental

Tool Shortened

Environmental

Checklist: Fitness

Centre Survey

(HEZEAT-FCS).59

Construct & justification

Developed by the investigators

of a community-based

participatory action research

study (still in progress) for

their participants with mobility

impairments to determine

accessibility and usability of

community fitness centers.59

Participants with

mobility

impairments in

research study.59

43 objective items taken directly from the AIMFREE

Fitness Centre Survey Consumer Survey within the

domains of: parking; entrance areas; locker rooms;

showers; equipment; programs; elevators; bathrooms;

and staff.

Items are scored yes, no or not applicable. The users’ also

complete an overall subjective rating (1e7 point VAS) of

fitness facility accessibility. No further information is

available to date regarding analysis and interpretation of

the scores.

Instrument available on-

line at no cost.59

Administered on hard

copy.

Length of time to

complete survey is not

known.

Currently being used in

participatory action

research study.59

Community Health

Environment Checklist

e Exercise Facilities

(CHEC-Fit).60

Construct & justification

As for CHEC-M.

Any person with an

interest in

accessibility of

fitness and

recreational sports

centers.

Accessibility and usability constructs are assessed over 54

items across four sections: main fitness area;

participation in exercise; locker rooms; miscellaneous

accommodations (e.g. fees, transportation, drinking

fountains).

Each item is marked yes, no or not applicable and these

terms have been allocated a corresponding score

(yes 5 1, no 5 0, not applicable 5 9). Although the

CHEC-Fit rule book clearly states CHEC-Fit is to be

used in conjunction with CHEC-M, it is not clear how

this is so.

Instrument available on-

line as a read only

document.60 Contact

instrument developers

to access.

Length of time to

complete survey is not

known.

CHEC-Fit to date has not

been applied in any

research studies.

31

A.M

.Calder

andH.F.

Mullig

an/D

isability

andHealth

Journal7(2014)26e

35

Fig. 2. Consensus-based Standards for the selection of health Measurement INstruments (COSMIN) measurement property appraisal graph.

32 A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

reliability, validity and responsiveness.74 Utilizing instru-ments that lack psychometric analysis, or those that demon-strate poor reliability and validity, means the user runsa high risk of obtaining inaccurate, biased and untrust-worthy information.42,53 In reality however, not everymeasurement property for a particular instrument will havebeen or needs to undergo psychometric evaluation.39e41,74

For example, psychometric evaluation is not consideredapplicable for the measurement properties of measurementerror, hypothesis testing, criterion validity and responsive-ness for discriminative instruments.74 However, otheraspects of reliability and validity for discriminative instru-ments are particularly important to evaluate psychometri-cally as they give insight regarding potential problems foradministration and interpretation. A recent systematicreview by Gray, Zimmerman and Rimmer75 that investi-gated built environment instruments for walkability, bike-ability and recreation found that only 53% of theinstruments contained psychometric information and onlyhalf of these demonstrated acceptable reliability. This isof concern because any instrument showing poor reliabilitywill obtain misleading results. Incomplete reporting ofinformation within studies and small sample sizes are

Table 3

Definitions of psychometric measurement properties according to the Consensus

(COSMIN) developers43(p743)

Domain Measurement property Definition

Reliability Internal consistency ‘‘The degree of inter-relatedness a

Testeretest reliability ‘‘The extent to which scores for p

over time’’.

Inter-rater reliability ‘‘The extent to which scores for p

by different persons on the sam

Validity Content validity ‘‘The degree to which the content

be measured’’.

Face validity ‘‘The degree to which the items o

reflection of the construct to be

Structural validity ‘‘The degree to which the scores o

of the construct to be measured

common problems encountered by researchers analyzingstudies for psychometrics.41 Of the seven instruments re-viewed in this study, only the AIMFREE and CHEC-Mhad undergone psychometric evaluation. However, whilenot every psychometric property has been evaluated forthese two instruments, it is concerning that some aspectsof these instruments scored only fair or poor (AIMFREEe structural validity and testeretest reliability; CHEC-Me inter-rater reliability and internal consistency). Usinginstruments with low reliability scores to evaluate fitnessand recreational sports centers means the instrument couldshow a high degree of error for repeated measurementsacross time (testeretest) and by different raters on the sameoccasion (inter-rater). These results therefore leave usunable to recommend an instrument of choice.

Determining the most useful instrument is a trade-offbetween sound psychometric qualities and the instrument’sfeasibility or ease of use. One aspect of feasibility that isparamount to the user is the time and effort required tocomplete the instrument (i.e. its respondent burden).CHEC-M has undergone some psychometric analysis(internal consistency, inter-rater reliability and content val-idity) and demonstrates low respondent burden (65 items).

-based Standards for the selection for health Measurement INstruments

mong items’’.

atients who have not changed are the same for repeated measurements

atients who have not changed are the same for repeated measurements

e occasion’’.

of an HR-PRO instrument is an adequate reflection of the construct to

f an HR-PRO instrument indeed looks as though they are an adequate

measured. Face validity is as aspect of content validity’’.

f an HR-PRO instrument are an adequate reflection of the dimensionality

. Structural validity is an aspect of construct validity.’’

33A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

While the HEZEAT-FCS and CHEC-Fit instruments alsoexhibit low respondent burden, they are however, too‘‘new’’ (published 2009 and 2010 respectively) to havebeen evaluated psychometrically. The opposite holds forthe AIMFREE instrument. Similar to CHEC-M, the AIM-FREE has undergone psychometric analysis for internalconsistency (and IRT), intra-rater reliability, content valid-ity and structural validity, but contains between 200 and422 items (depending on which version is used), therebyconsiderably increasing respondent burden. AIMFREEinstrument users have reported it can take between twoand three hours to complete the survey.36,63,66 However,to address the issue of respondent burden, the AIMFREEinstrument developers are currently working on a shortenedversion called the AIMFREE CAT.63 Using computer adap-tive testing (CAT) technology, developers have shown asfew as 30 items from the original AIMFREE instrumentcan accurately evaluate a fitness facility’s accessibility.63

Interpretation of the scores from instruments requiresconsideration. Problems arise when instruments sumdichotomous ordinal scores to gain an overall accessibilityscore. This is because one cannot assume that raw dichoto-mous data are all linear.71 Bendal22 highlights this issue,noting that if all raw scores were seen as linear, it wouldbe possible for a facility to obtain a high overall accessi-bility score even if it was not accessible to wheelchair usersbecause of narrow entrance ways. If the instrument userwishes to determine an overall accessible rating, theywould need to choose an instrument whose scoring algo-rithm accommodates non-linearity of dichotomous ordinaldata. The AIMFREE instrument is the only instrument inour review that has done this. The authors of the AIMFREEinstrument applied the Rasch measurement model at theinstrument’s development phase.4,57 Rasch analysis re-calibrates raw non-linear data and transforms it into equalinterval units to create a standardized and reproduciblemeasurement scale.69,70,72,73 Users may wish to select theAIMFREE instrument over other instruments because ofits Rasch scoring algorithm.

This review has limitations. Only two of the instrumentsincluded in the review showed some psychometric analysis.Three other instruments (RBHCFF, HEZEAT-FCS andCHEC-Fit) are possibly too new to have undergone psycho-metric evaluation. Additionally, the COSMIN checklistmay carry limitations. The use of the ‘‘worse score counts’’(as prescribed by the COSMIN developers) has potentiallimitations in identifying item importance. Therefore, theinstrument may benefit from undergoing an item responsetheory (IRT) analysis (for example, Rasch analysis).

Conclusion

This systematic review identified and evaluated seveninstruments that assess accessibility of fitness and recrea-tional sports centers and described their qualitative attri-butes and quantitative properties. Only the AIMFREE and

CHEC-M instruments have undergone psychometric evalu-ation. Based on our review, we found that, while bothscored excellent for content validity, some aspects of theirpsychometric properties scored as fair/poor or could notbe assessed. In addition, only the AIMFREE instrumenthas developed a scoring algorithm that accommodatesnon-linearity of dichotomous ordinal data. This means thatother instruments may provide misleading information tothe user. An important future consideration for instrumentdevelopment is that developers conduct full psychometricassessment of their instrument using adequate sample sizes.In addition, we recommend that they carefully considertheir method of scoring. Lastly, we recommend consider-ation of respondent burden so that users are provided withscientifically robust instruments that are feasible toadminister.

Acknowledgments

We wish to thank colleagues Mrs Emma Hewat as thesecond independent reviewer appraising the measurementproperties of the AIMFREE and CHEC-M instrumentsand Dr Margot Skinner for editing a draft of this manu-script.

References

1. Nary D, Froehlich K, White G. Accessibility of fitness facilities for

persons with physical disabilities using wheelchairs. Top Spinal Cord

Inj Rehabil. 2000;6(1):87e98.

2. Rimmer J. Measuring the accessibility of fitness and recreation envi-

ronments. Neurorehabil Neural Repair. 2005;19(1):37S.

3. Rimmer J, Braddock D, Petetti K. Research on physical activity and

disability: an emerging national priority. Med Sci Sports Exerc.

1996;28(8):1366e1372.

4. Rimmer J, Riley B, Wang E, Rauworth A. Accessibility of health

clubs for people with mobility disabilities and visual impairments.

Am J Public Health. 2005;95(11):2022e2028.

5. Rimmer J, Riley B, Wang E, Rauworth A, Jurkowski J. Physical

activity participation among persons with disabilities: barriers and

facilitators. Am J Prev Med. 2004;26(5):419e425.

6. Rimmer J. Health promotion for individuals with disabilities: the

need for a transitional model in service delivery. Dis Manag Health

Outcome. 2002;10(6):337e343.

7. Rimmer J. The conspicuous absence of people with disabilities in

public fitness and recreation facilities: lack of interest or lack of

access? Am J Health Promot. 2005;19(5):327e329.8. Santiago M, Coyle C. Leisure-time physical activity and secondary

conditions in women with physical disabilities. Disabil Rehabil.

2004;26:485e494.

9. Rimmer J, Rubin S, Braddock D. Barriers to exercise in African

American women with physical disabilities. Arch Phys Med Rehabil.

2000;81:182e188.

10. US Census Bureau. North American Industry Classification System

Code (NAICS) 713940. [cited 2012 10 May]. Available from: http://

www.census.gov/epcd/ec97/def/713940.HTM; 2012.

11. Rimmer J. Health promotion for people with disabilities: the emerging

paradigm shift from disability prevention to prevention of secondary

conditions. Phys Ther. 1999;79(5):495e502.

12. Elsworth C, Dawes H, Sackley C, et al. A study of perceived facili-

tators to physical activity in neurological conditions. Int J Ther Re-

habil. 2009;16(1):17e23.

34 A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

13. Froehlich K, Nary D, White G. Identifying barriers to participation in

physical activity for women with disabilities. SCI Psychosocial

Process. 2002;15:21e29.

14. Motl R, Snook E, McAuley E, Scott J, Douglass M. Correlates of

physical activity among individuals with multiple sclerosis. Ann Be-

hav Med. 2006;32:154e161.

15. Rimmer J, Wang E, Smith D. Barriers associated with exercise and

community access for individuals with stroke. J Rehabil Res Dev.

2008;45(2):315e322.16. Scelza W, Kalpakjian C, Zemper E, Tate D. Perceived barriers to

exercise in people with spinal cord injury. Am J Phys Med Rehabil.

2005;84(8):576e583.17. Vanner E, Block P, Christodoulou C, Horowitz B, Krupp L. Pilot

study exploring quality of life and barriers to leisure-time physical

activity in persons with moderate to severe multiple sclerosis. Disabil

Health J. 2008;1:58e65.18. Iwarsson S, Stahl A. Accessibility, usability and universal design e

positioning and definition of concepts describing person-environment

relationships. Disabil Rehabil. 2003;25(2):57e66.

19. Rolfe D, Yoshida K, Renwick R, Bailey C. Beyond barriers: women

living with disabilities and their participation in community exercise

programs [cited 2011 10 November]. Available from: http://stargate.

uwaterloo.ca/~jzelek/teaching/syde361/ficcdat2007/Full%20Papers/

GOWD/G0052.pdf; 2007.

20. Ministry of Health. The New Zealand Disability Strategy. Wellington:

Ministry of Health [cited 2010 28 April]. Available from: www.odi.

govt.nz/resources/publications/nz-disability-strategy.pdf; 2001.

21. WelageN,LiuK.Wheelchair accessibility of public buildings: a review

of the literature. Disabil Rehabil Assist Technol. 2011;6(1):1e9.

22. Bendel J. Decision support system for evaluating accessibility of

facilities. Israeli J Occup Ther. 2006;15(3).

23. Ramot A, Lahav Y, Bendel J. Planning accessibility in the old city of

Jerusalem. In: Preiser W, Smith K, eds. Universal Design Handbook.

McGraw Hill; 2010. p. 16.1e16.11.24. Access Board. Americans with Disabilities Act Accessibility Guidelines

(ADAAG) Checklist for Buildings and Facilities [cited 2010 17 April].

Available from: http://www.access-board.gov/adaag/checklist/a16.html;

1992.

25. Bullock C, Mahon M. Introduction to Recreation Services for People

with Disabilities: A Person-Centered Approach. 2nd ed. United

States: Sagamore; 2001.

26. Hiss B, Rauworth A. Creating an Accessible Fitness or Wellness

Facility [cited 2010 25 March]. Available from: www.ncpad.org/

get/pdf/hiss_rauworth_2007.pdf; 2007.

27. North Carolina Office on Disability and Health. Removing Barriers

to Health Clubs and Fitness Facilities. A Guide for Accommodating

All Members, Including People with Disabilities and Older Adults

[cited 2010 15 April]. Available from: http://www.fpg.unc.edu/

~ncodh/pdfs/rbfitness.pdf; 2008.

28. Cardinal B, Spaziani M. ADA compliance and the accessibility of

physical activity facilities in Western Oregon. Am J Health Promot.

2003;17(3):197e201.

29. Edwards K, Finco K, Foss S, Johnson M, Snyder J, Stoelzle H. ADA

Compliance and Accessibility of Fitness Facilities in Western Wiscon-

sin [cited 2012 13 April]. Available from: http://minds.wisconsin.edu/

bitstream/handle/1793/47399/EdwardsSpr10.pdf; 2010.

30. Pike H, Walker J, Collins J, Hodges J. An investigation of ADA

compliance of aquatic facilities in the North Texas area. Am J Health

Promot. 2008;23(2):139e146.

31. Figoni S, McClain L, Bell A, Degnan J, Norbury N, Rettle R. Acces-

sibility of fitness facilities in the Kansas City metropolitan area. Top

Spinal Cord Inj Rehabil. 1988;3(3):66e78.

32. Bendel J, Klupfel H. Accessibility and Evacuation Planning e Similarities

and Differences [cited 2011 25 November]. Available from: http://pedgo.

com/downloads/pedrestrians/downloads/publications/Bendel2010.pdf; 2010.

33. Riley B, Rimmer J, Wang E, Schiller W. A conceptual framework

for improving the accessibility of fitness and recreation facilities

for people with disabilities. J Phys Activity Health. 2008;5:

158e168.

34. Access Board. Federal Register. Part II: Architectural and Transpor-

tation Barriers Compliance Board. National Archives and Records

Administration; 2002.

35. Adaptive Environments Centre Inc. The Americans with Disabilities

Act Checklist for Readily Achievable Barrier Removal. ADA Title III

Compliance Materials. version 2.1.[cited 2011 12 August]. Available

from: http://www.wbdg.org/resources/assesstoolsphp?r5productive;

1995.

36. Rectech. Online Environmental Tool for Fitness and Recreation

Facilities. University of Illinois, Chicago. [cited 2011 15 August ].

Available from: http://www.rectech.org/inner.php?sheet56; 2009.

37. French D, Hainsworth J. ‘There aren’t any buses and the swimming

pool is always cold!’: obstacles and opportunities in the provision of

sport for disabled people. Managing Leis. 2001;6:35e49.38. Ministry of Social Development. Built Environment e Office for

Disability Issues. Wellington: Ministry of Social Development. [cited

2010 20 April]. Available from: www.odi.govt.nz/what-we-do/built-

environment/index.html.

39. Hobart J. Evaluating neurological outcome measures: the bare essen-

tials. J Neurol Neurosurg Psychiatry. 1996;60:127e130.

40. Johnston M, Graves D. Towards guidelines for evaluation of

measures: an introduction with application to spinal cord injury.

J Spinal Cord Med. 2008;31:13e26.

41. Terwee C, Mokkink L, van Poppel M, Chinapaw M, van

Mechelen W, de Vet H. Qualitative attributes and measurement prop-

erties of physical activity questionnaires. A checklist. Sports Med.

2010;40(7):525e537.

42. Mokkink L, Terwee C, Patrick D, et al. The COSMIN checklist for

assessing the methodological quality of studies on measurement

properties of health status measurement instruments: an international

Delphie Study. Qual Life Res. 2010;19:539e549.

43. Mokkink L, Terwee C, Patrick D, et al. The COSMIN study reached

international consensus on taxonomy, terminology, and definitions of

measurement properties for health-related patient-reported outcomes.

J Clin Epidemiol. 2010;63:737e745.

44. Chinapaw M, Mokkink L, van Poppel M, van Mechelen W,

Terwee C. Physical activity questionnaires for youth. A systematic

review of measurement properties. Sports Med. 2010;40(7):539e563.

45. Forsen L, Loland N, Vuillemin A, et al. Self-administered physical

activity questionnaires for the elderly. A systematic review of

measurement properties. Sports Med. 2010;40(7):601e623.

46. Lubans D, Hesketh K, Cliff D, et al. A systematic review of the val-

idity and reliability of sedentary behaviour measures used with chil-

dren and adolescents. Obestiy Rev. 2011;12:781e799.47. Rydwik E, Bergland A, Forsen L, Frandin K. Psychometric proper-

ties of timed up and go in elderly people: a systematic review. Phys

Occup Ther Pediatr. 2011;29(2):102e125.48. Rydwik E, Bergland A, Forsen L, Frandin K. Investigation into the

reliability and validity of the measurement of elderly people’s

walking speed: a systematic review. Physiother Theor Pract.

2012;28(3):238e256.

49. Terwee C, Bouwmeester W, van Elsland S, de Vet H, Dekker J.

Instruments to assess physical activity in patients with osteoarthritis

of the hip or knee: a systematic review of measurement properties.

Osteoarthritis Cartilage. 2011;19:620e633.

50. van Poppel M, Chinapaw M, Mokkink L, van Mechelen W,

Terwee C. Physical activity questionnaires for adults. A systematic

review off measurement properties. Sports Med. 2010;40(7):

565e600.

51. Terwee C, Mokkink L, Knol D, Ostelo R, Bouter L, de Vet H. Rating

the methodological quality in systematic reviews of studies on

measurement properties: a scoring system for the COSMIN checklist.

Qual Life Res. 2012;21(4):651e657.

52. Mokkink L, Terwee C, Gibbons E, et al. Inter-rater reliability of the

COSMIN (COnsensus-based Standards for the selection of health

35A.M. Calder and H.F. Mulligan / Disability and Health Journal 7 (2014) 26e35

status Measurement Instruments) Checklist. BMC Med Res Method-

ol. 2010;10:82.

53. Mokkink L, Terwee C, Knol D, et al. The COSMIN checklist for

evaluating the methodological quality of studies on measurement

properties: a clarification of its content. BMC Med Res Methodol.

2010;10:22.

54. Rimmer J, Gray-Stanley J, Haugen B. Examination of Instruments

Used toMeasure theBuilt Environment andPhysical Activity. Chicago:

University of Illinois: Department of Disability and Human Develop-

ment; 2009.

55. The University of Illinois. Universal Design and Health Promotion.

Chicago: The University of Illinois: Centre on Health Promotion

Research for Persons with Disabilities [cited 2010 18 April]. Avail-

able from: http://www.uic-chp.org/CHP_A9_UDHP_01.html; 2009.

56. Rimmer J, Riley B. Accessibility Instruments Measuring Fitness and

Recreation Environments: Fitness Centre and Swimming Pool Acces-

sibility. Professional Version. Chicago: RERC RecTech; 2004e2006.

57. Rimmer J, Riley B, Wang E, Rauworth A. Development and valida-

tion of AIMFREE: accessibility instruments measuring fitness and

recreation environments. Disabil Rehabil. 2004;26(18):1087e1095.58. Stark S, Hollingsworth H, Morgan K, Gray D. Development of

a measure of receptivity of the physical environment. Disabil Reha-

bil. 2007;29(2):123e137.59. The University of Illinois. Health Empowerment Zone Environmental

Tool Shortened Environmental Checklist: Fitness Centre Survey [cited

2009 5 May]. Available from: http://www.uic-chp.org/ARTICLES/

HEZ/Ennvironmental%20Assessments/Fitness%20Center%20Check-

list.pdf; 2009.

60. Disability and Community Participation Research Office. Community

Participation and Receptivity. St Louis: Washington University [cited

2011 10 November]. Available from: http://communityparticipation.

wustl.edu/surveys/CHEC.html; 2010.

61. Hamzat T, DadaO.Wheelchair accessibility of public buildings in Iba-

dan, Nigeria. Asia Pacific Disabil Rehabil J. 2005;16(2):115e124.

62. Rivano-Fischer D. Wheelchair accessibility of public buildings in Al

Ain, United Arab Emirates (UAE). Disabil Rehabil. 2004;26(19):

1150e1157.

63. Rimmer J, Zhu W, Padalabalanarayanan S, Boiarskaia E, Lullo C,

eds. Online Environmental Assessment Tool and Use of Computer

Adaptive Testing (CAT) for Achieving Optimization of Fitness and

Recreation Facility Accessibility. State of the Science Conference;

2010.

64. Peterson J, Baylor Uni Waco T, Piletic C, eds. An Anlaysis of

Community Based Not-for-Profit Fitness Centre Accessibility.

2011 AAHPERD National Convention and Exposition; 2011

Mar 29.

65. Rauworth A, Rimmer J, Wang E, Riley B. A national study of assess-

ing the accessibility of fitness facilities for people with disabilities.

Med Sci Sports Exerc. 2003;35(5):S65.

66. Arbour-Nicitopoulos K, Ginis K. Universal accessibility of ‘‘accessi-

bile’’ fitness and recreational facilities for persons with mobility

disabilities. Adapt Phys Activity Q. 2011;28:1e15.

67. Aldimkhi B. Accessibility of Health Clubs and Fitness Centres for

People with Disabilities in Kuwait. New Britain: Central Connecticut

State University; 2009.

68. Koh R. Accessibility of Community Sports and Recreation Facilities

for People with Physical Disabilities in Singapore (Abstract). Na-

nyang Technological University; 2009.

69. Chang C, Reeve B. Item response theory and its applications to

patient-reported outcomes measurement. Eval Health Professions.

2005;28(3):264e282.

70. Reeve B. Item response theory modeling in health outcomes

measurement. Expert Rev Pharmacoecon Outcomes Res. 2003;3(2):

131e145.

71. Tatum D. Rasch analysis: an introduction to objective measurement.

Lab Med. 2000;31(5):272e274.72. Tennent A, Conaghan P. The Rasch measurement model in rheuma-

tology: what is it and why use it? When should it be applied, and

what should one look for in a Rasch paper? Arthritis Rheum.

2007;57(8):1358e1362.73. Furr R, Bacharach V. Item Response Theory and Rasch Models.

Psychometrics: An Introduction. Sage; 2007. p. 314e334.

74. Greenhalgh J, Long A, Brettle A, Grant M. Reviewing and selecting

outcome measures for use in routine practice. J Eval Clin Pract.

1998;4(4):339e350.

75. Gray J, Zimmerman J, Rimmer J. Built environment instruments for

walkability, bikeability, and recreation: disability or universal design

relevant? Disabil Health J. 2012;5:87e101.