2010; 32: 947–952

BEME RAPID REVIEW

Measuring the educational environment inhealth professions studies: A systematic review

DIANTHA SOEMANTRI1, CRISTIAN HERRERA2,3 & ARNOLDO RIQUELME2,4

1Department of Medical Education, Faculty of Medicine, Universitas Indonesia, Indonesia, 2School of Medicine, PontificiaUniversidad Catolica de Chile, Chile, 3Evidence Based Medicine Unit, Pontificia Universidad Catolica de Chile, Chile,4Department of Gastroenterology, Pontificia Universidad Catolica de Chile, Chile

Abstract

Background: One of the determinants of the medical student’s behaviour is the medical school learning environment.

Aim: The aim of this research was to identify the instruments used to measure the educational environment in health professions

education and to assess their validity and reliability.

Methods: We performed an electronic search in the medical literature analysis and retrieval system online (MEDLINE) and Timelit

(Topics in medical education) databases through to October 2008. The non-electronic search (hand searching) was conducted

through reviewing the references of the retrieved studies and identifying the relevant ones. Two independent authors read, rated

and selected studies for the review according to the pre-specified criteria. The inter-rater agreement was measured with Kappa

coefficient.

Results: Seventy-nine studies were included with the Kappa coefficient of 0.79, which demonstrated a reliable process, and 31

instruments were extracted. The Dundee Ready Education Environment Measure, Postgraduate Hospital Educational Environment

Measure, Clinical Learning Environment and Supervision and Dental Student Learning Environment Survey are likely to be the

most suitable instruments for undergraduate medicine, postgraduate medicine, nursing and dental education, respectively.

Conclusions: As a valid and reliable instrument is available for each educational setting, a study to assess the educational

environment should become a part of an institution’s good educational practice. Further studies employing a wider range of

databases with more elaborated search strategies will increase the comprehensiveness of the systematic review.

Introduction

One determinant of a medical student’s behaviour is the

medical school learning environment (Genn 2001a). The

environment related to the student’s behaviour is as perceived

by the student. Genn and Harden (1986) suggested that

although the concept is rather intangible, the effects of an

educational environment are substantial, real and influential.

The climate makes a notable contribution to the prediction of

student achievement, satisfaction and success (Genn 2001b).

The relationship between the educational environment as

perceived or experienced by the students and their achieve-

ment, satisfaction and success in medical school has brought

into awareness the need to study or evaluate the medical

school’s environment. The World Federation for Medical

Education (1998) has considered the learning environment as

one of the targets for the evaluation of medical education

programmes. Based on all of the above, a study of the

environment should become an inseparable part of the study

of a medical school’s curriculum.

Genn (2001a) provides a definition for educational envi-

ronment as the manifestation of a curriculum. Rothman and

Ayoade (1970) also consider the learning environment as ‘a

manifestation of the effects on students of the various parts of

the curriculum’. Maudsley (2001) states that ‘a learning

environment exists wherever and whenever students

gather . . . ’ and it includes various factors, which contribute

to an effective education and becomes the background in

which the curriculum exists.

Pace and Stern (1958) suggest that an educational environ-

ment is characterized by the pressures, stresses, practices,

policies, rewards and values within the classroom or school of

which they refer to as ‘environmental press’. Furthermore, one

of the 10 questions to be asked when planning a course or

curriculum is ‘what educational environment or climate should

be fostered?’ (Harden 1986). The course or curriculum planner

Practice points

. Measuring the educational environment should become

a part of educational institutions’ good practices.

. Each educational context or setting demands different

aspects of environment to be measured.

. Various instruments to measure educational environ-

ment in different health professions education settings

are available.

. Some instruments have proven their psychometric

values.

Correspondence: Diantha Soemantri Department of Medical Education, Faculty of Medicine, Universitas Indonesia, Jakarta Pusat 10430, Indonesia.

Tel/Fax: 62 21 3901814; email: dianthasoemantri@yahoo.com; diantha.soemantri@ui.ac.id

ISSN 0142–159X print/ISSN 1466–187X online/10/120947–6 � 2010 Informa UK Ltd. 947DOI: 10.3109/01421591003686229

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Maa

stri

cht o

n 06

/18/

14Fo

r pe

rson

al u

se o

nly.

should evaluate those areas that are fostered and encouraged

in the institution.

The term environment and climate are often used inter-

changeably in educational literature. Genn (2001b) defines the

term climate as the educational environment of an institution

as perceived by students. Roff and McAleer (2001) use the term

educational climate as the synonym of educational environ-

ment. In the study conducted by Rothman & Ayoade (1970),

the learning environment is defined as what the students

perceive and thus no differentiation is made between the term

environment and climate. Therefore, in this article the educa-

tional climate will be regarded as similar to educational

environment.

However, it is the students’ perceptions of the environment,

rather than the environment itself, that determines the behav-

iour (Hutchins 1961; Rothman & Ayoade 1970; Genn 2001a;

Konings et al. 2005). Students’ perceptions of the classroom

environment have been used to predict their outcomes (Fraser

et al. 1986). Students, as the ‘inhabitants’ of the (Genn 2001a),

are the ones who experience the environment and it is their

perceptions of the environment, which then becomes one of

the determinants of their behaviour.

A quantitative measurement of the educational environ-

ment requires the use of an inventory or instrument. The

selection of such an instrument should be based on the

qualities of the measurement process using a particular

instrument or the suitability of that instrument to measure

educational environment. The qualities, or often referred to as

psychometric features, usually come under two main head-

ings, which are validity and reliability. A valid and reliable

learning environment tool allows a meaningful measure of the

learning environment of an institution, and thus appropriate

measures to improve the environment can be taken. The

differences among various educational settings may require a

different educational environment inventory, which suits the

specific situation of a particular institution.

In any measurement process, it is necessary to ascertain

whether the instrument or inventory measures what it

supposes to measure; and validity deals with this specific

question. Content validity refers to the extent that the

instrument or inventory measures the intended subject

matter content (Gronlund 1976a). Criterion-related validity

refers to the extent to which the measurement results can be

used to predict a future outcome and also the extent to which

they correlate with current results (performance) obtained by

other valid measures (Gronlund 1976a). Construct validity may

be defined as the extent to which the test can be used to

measure certain psychological constructs and the results

interpreted in terms of those constructs (Gronlund 1976a).

Reliability refers to the reproducibility of the measurement

or assessment results. A result may be reliable over different

periods of time, over different raters or over different samples

of questions (Gronlund 1976b). An unreliable or inconsistent

measurement result cannot possibly permit valid interpretation

of the result. A high correlation between the scores of

individual items, in other words a high internal consistency,

would indicate that the scores measure a single construct. The

degree of reliability adequate for a particular measurement

depends on: the purpose of the measurement, the use of the

results, the importance of the decision that will be made and

ultimately the consequences resulting from the measurement.

Reliability coefficient of 0.60 is considered acceptable for

questionnaires (Nunnally 1978).

Objectives

The objectives of this systematic review were to respond to the

following two research questions:

(1) Which instruments or inventories are used to measure

the educational environment in health professions

education?

(2) How is the suitability of each instrument in measuring

the educational environment in health professions

education?

Methods

A systematic review approach is applicable in medical

education research. The purpose is similar to that of the

systematic review in medicine, which is to reflect the best

evidence available in medical education (BEME Collaboration

2006). Steps in conducting a systematic review can be

summarized as follows: formulation of a specific review

question, comprehensive search and inclusion of studies,

quality assessment of included studies and data extraction,

synthesis of study results, interpretation of results and report

writing (Pai et al. 2004). These steps should result in a

summary of evidence related to a specific research question.

Search strategy

We performed an electronic search in the MEDLINE and

Timelit (Topics in Medical Education) databases through to

October 2008. The non-electronic search (hand searching) was

conducted through reviewing the references of the retrieved

studies and identifying the relevant ones. The MEDLINE search

strategy combined the available and appropriate headings

medical subject (MeSH) which are (exp) educational mea-

surement/, (exp) education/, (exp) learning/ with the free

keywords ‘education* environment’, ‘climate’, ‘learning envi-

ronment’ and ‘education* climate’. In the Timelit database in

which the search using subject headings is not possible, a

search using the relevant keywords (‘educational climate’,

‘educational environment’, ‘education environment’ and

‘learning environment’) was conducted.

Inclusion and exclusion criteria

We included studies that were in the area of health professions

education (undergraduate or postgraduate), which measures

the educational or learning environment with a quantitative

method (employs an instrument or inventory). A study was

excluded if it was a review article, a study in non-health

professions education, did not measure the overall educational

or learning environment of the institution and employed a

qualitative research method.

D. Soemantri et al.

948

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Maa

stri

cht o

n 06

/18/

14Fo

r pe

rson

al u

se o

nly.

Data extraction

One author read all the titles retrieved by the searches and

eliminated the studies that were clearly of another subject.

From the resultant list, two independent authors read the

abstracts in the records to identify publications that appeared

to be eligible for this review. From the chosen abstracts, two

independent authors read the full texts, rated and selected

studies for the review according to the pre-specified (inclusion

and exclusion) criteria. Disagreements were resolved by

discussion. The inter-rater agreement was measured with

Kappa coefficient.

Data synthesis

To respond to the first question of this review, the search

strategy allowed us to identify the inventories or instruments

that were used to measure the educational environment in

health professions education.

To answer the second research question, the data regarding

the validity and reliability of the educational environment

measurement results using each particular inventory was

extracted, analyzed and synthesized. Some instruments were

used in several studies, therefore the information was gathered

from all the included studies utilizing those particular instru-

ments. Based on the available information, the most suitable

instrument or inventory to measure educational environment

in different fields of health professions education was identi-

fied. The information concerning the educational environment

measurement instrument was extracted from the included

studies: the health professions education setting in which the

instrument was developed and/or used; the general informa-

tion about the instrument, including its subscales; the validity

of the results (content validity, criterion-related validity,

construct validity) and the reliability of the results.

Results

Following the search strategy, the electronic databases

(MEDLINE 1966 to October 2008 and Timelit) yielded 951

titles. Duplication of titles from the two databases were

identified and removed. As a result of the search strategy and

subsequent screening, 178 studies (1958–2008) were consid-

ered to be relevant for further review. Full papers of the studies

were obtained. Two reviewers worked independently to

review the studies and make the decision on each of them.

The reviewers disagreed on 19 studies. Therefore, the inter-

rater agreement was substantial, with 90% agreement and the

Kappa coefficient of 0.79. Kappa coefficient takes into account

the occurrence of chance agreement.

Thirty one studies were excluded because they were not

primary studies of educational environment. Those studies

were literature reviews, editorials or commentaries/letters

around the topic of educational environment. Another exclu-

sion criterion was the primary educational environment study,

which was conducted in non-health professions education

settings, such as secondary education or other subject disci-

plines in higher education level, 14 studies were excluded

following this criterion.

Eighteen studies employed qualitative research methods,

such as interviews or observation, to assess the educational

environment or certain aspects of it. Thus, those studies were

excluded. Another 36 studies were excluded because they

only measured a certain aspect of the educational environ-

ment, such as students’ stress, coping, support services,

student abuse and harassment, perceptions of teaching and

course experience. The reviewers considered those studies to

have too narrow coverage of educational environment, and

therefore could not be regarded as measuring the full

educational environment.

Following the flowchart, 79 studies from 1961 to 2008 were

included (Figure 1). These were primary studies, which

measured the educational environment in health professions

education using specific instruments. The area of health

professions education included undergraduate and postgrad-

uate medicine, nursing and dental education.

Thirty one instruments were identified from the 79 included

studies. Some instruments were developed specifically to be

used in particular settings, while others were adapted from

another instrument. Ten instruments were used in undergrad-

uate medical education, seven instruments for postgraduate

medical education, eight and four instruments were used in

nursing and dental education, respectively. One instrument

was used for both undergraduate and postgraduate medical

education and one instrument was used in all areas (under-

graduate and postgraduate medicine, nursing, dental, chiro-

practic education). Table 1 summarizes the identified

instruments along with the settings in which the instruments

were used.

Data regarding the use of the identified instruments in each

article were extracted and synthesized. They include the

psychometric properties presented in the articles as the results

of the use of the instruments in the educational environment

Studies on educational or learningenvironment (178)

Review studies (31)Studies in non health

professions education (14)

Original undergraduate andpostgraduate studies (133)

Studies with qualitativemethods (18)

Studies with quantitative method (79)(Kappa coefficient=0.79)

Instruments which measure educational environment in health professions education (31)

Studies which measure onlya particular aspect of the

environment (36)

Figure 1. Disposition of 178 articles on educational/learning

environment.

Note: Solid and dash lines indicate included and excluded

studies, respectively.

BEME Rapid Review

949

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Maa

stri

cht o

n 06

/18/

14Fo

r pe

rson

al u

se o

nly.

measurement processes. The synthesis of data were presented

in the form of table for each area of the health profession study

(Tables 2–5, available at www.medicaltacher.org). The row

lists the educational environment measurement instrument

whereas the column lists the psychometric properties of each

instrument, which consist of content validity/content evidence,

criterion validity/relation other variables, construct validity and

reliability/internal structure.

Discussion

Specific databases which contain publications in medical

education are of limited availability. It becomes more chal-

lenging to conduct a comprehensive search because of this

issue. Relevant papers are more likely to be missed and large

numbers of false hits occur. Moreover, finding the appropriate

subject headings may become a problem because the database

(MEDLINE) is not specific to medical education and thus, the

availability of subject headings relevant to medical education is

limited. It was then necessary to combine the search with

relevant keywords (‘education environment’, ‘education cli-

mate’, ‘learning environment’ and ‘climate’) to increase the

specificity. Notwithstanding, our search strategy was likely to

mislay relevant studies due to the lack of specific search tools

for the medical education area. In that order, the use of only

two databases and a low-sensibility search strategy become

limitations of this study. This is caused by the limited resources

on the researcher’s side. For a systematic and comprehensive

search to be successfully conducted, adequate resources

(human resources, educational material, time and cost) are

essential. In order to improve the comprehensiveness of the

search, the hand searching of the references cited in the

retrieved articles was conducted. The process yielded a

substantial number of additional articles, which were not

retrieved through the database search.

The decision to include or exclude each study was based on

the criteria visualized in the flowchart (Figure 1). Although the

flowchart was clear and the reviewers understood the criteria

beforehand, some disagreements occurred inadvertently

during the process. The disagreements mostly emerged when

making the decisions on studies that only measured a certain

aspect of the educational environment; they resulted from

differences in interpretation about those particular studies.

A high percentage of agreement (90%) between the two

independent reviewers and the Kappa coefficient of 0.79 show

a fairly reliable result of the inclusion/exclusion (review)

process. The systematic search yielded a substantial number of

instruments for measuring educational environment in health

professions education. The measurement of educational envi-

ronment has received important attention from educationalists,

highlighted by the availability of various instruments to be

utilized in different educational settings.

Analysis of the validity and reliability of educationalenvironment measurement instruments

Medicine (undergraduate). There were 12 educational envi-

ronment measurement instruments identified in this study.

Table 1. Undergraduate and postgraduate health profession educational environment measurement instruments identified inthe systematic review.

No. Instrument/Inventory Setting

1 Medical school learning environment Survey – modified (MSLES) Medicine

2 Learning environment questionnaire (LEQ) Medicine

3 Questionnaire from Parry et al. Medicine

4 Veteran affairs (VA) learners’ perceptions survey Medicine (postgraduate, PG)

5 Learning environment assessment (LEA) Medicine (PG)

6 Medical school environment questionnaire (MSEQ) Medicine

7 Questionnaire from Robins et al. Medicine

8 Medical school environment inventory (MSEI) Medicine

9 Questionnaire from Rotem, Godwin and Du Medicine (PG)

10 Course valuing inventory (CVI) Medicine

11 DREEM Medicine (undergraduate and PG), nursing,

dental and chiropractic

12 Operating room educational environment measure (OREEM) Medicine (PG)

13 Instrument from Pololi and Price Medicine

14 Surgical theatre educational environment measure (STEEM) Medicine (undergraduate and PG)

15 Anaesthetic theatre educational environment measure (ATEEM) Medicine (PG)

16 Practice-based educational environment measure Medicine (PG)

17 Postgraduate hospital educational environment measure (PHEEM) Medicine (PG)

18 Questionnaire from Patel and Dauphinee Medicine

19 Learning Climate Measure from Wangsaturaka Medicine

20 Questionnaire from Orton – Modified Nursing

21 Questionnaire from Hart and Rotem Nursing

22 Clinical learning environment scale (CLE) Nursing

23 Clinical learning environment inventory (CLEI) Nursing

24 College and university classroom environment inventory (CUCEI) Nursing

25 Clinical learning environment and supervision (CLES) Nursing

26 Clinical post-conference learning environment survey (CPCLES) Nursing

27 Postgraduate research experience questionnaire (PREQ) Nursing

28 Questionnaire from Gerzina, McLean and Fairley Dentistry

29 Clinical education instructional quality (ClinEd IQ) Dentistry

30 Learning environment survey (LES) Dentistry

31 Dental student learning environment survey (DSLES) Dentistry

D. Soemantri et al.

950

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Maa

stri

cht o

n 06

/18/

14Fo

r pe

rson

al u

se o

nly.

The summary of the psychometric qualities of the instrument-

sare presented in Table 2, available at www.medicalteacher.

org. Several instruments (questionnaire from Parry et al.

(2002), Pololi & Price (2000), Patel and Dauphinee (1985),

Robins et al. (1997), MSEI (Hutchins 1961) and MSEQ

(Wakeford 1981, 1984)) had considerable weaknesses in

their validity and reliability. The Learning Climate Measure

was an instrument developed specifically to measure the

educational environment of Thai medical education

(Wangsaturaka 2005). Therefore, it was a nation-specific

instrument and its use was limited to that specific context.

The STEEM was originally developed for postgraduate surgical

learning setting. When applied to the undergraduate setting, it

has demonstrated some weaknesses in terms of its validity

(Nagraj et al. 2006).

The LEQ (Rothman 1970; Rothman & Ayoade 1970; Levy

et al. 1973; Kaufman & Mann 1996; Schwartz & Loten 2003,

2004a, b) MSLES (Feletti & Clarke 1981a, b; Clarke et al. 1984;

Lancaster et al. 1997) and DREEM (Roff et al. 1997, 2001;

Al-Qahtani 1999; Al-Zidgali 1999; Baozhi 2003; Bassaw et al.

2003; Vieira et al. 2003; Zaini 2003; Al-Hazimi et al. 2004a, b;

Mayya & Roff 2004; Till 2004, 2005; Jiffry 2005; Varma et al.

2005; Dunne et al. 2006) demonstrated the robustness in terms

of their psychometric qualities. Their content validity was

established and they also had high internal consistency. The

construct validity of the MSLES and DREEM were indicated by

their ability to significantly differentiate the students’ percep-

tions of the learning environment between medical schools

with the traditional curriculum and ones with the more

innovative curriculum (Feletti & Clarke 1981b; Al-Qahtani

1999; Zaini 2003; Al-Hazimi et al. 2004a). The DREEM showed

an additional strength because it could be applied in medical

schools in different countries, cultures and contexts and still

demonstrated consistent reliability. In relation to criterion

validity, there was a relationship between the DREEM scores,

which reflected the students’ perceptions of the learning

environment and academic achievement (Baozhi 2003; Mayya

& Roff 2004).

According to the data presented in the related articles, the

DREEM is likely to be the most suitable instrument to be

applied in undergraduate medical education settings. The

validity had been established, and the instrument demon-

strated highly reliable results consistently throughout its

administration in different contexts. The results also indicated

the ability of the DREEM to differentiate between the learning

environment of the traditional and one of the more innovative

medical schools.

Medicine (postgraduate). Most of the identified educational

environment measurement instruments in postgraduate med-

ical education were developed for use in specific postgraduate

specialty areas. The psychometric qualities of the identified

instruments are summarized in Table 3, available at www.

medicalteacher.org. Postgraduate medical education pro-

grammes consisted of many different specialties and levels of

training. Each speciality had its own uniqueness in terms of the

educational environment. This situation often required a

specific instrument which could address the certain aspects

of the educational environment in the particular setting.

Four instruments: the STEEM (Cassar 2004), the OREEM

(Kanashiro et al. 2006), the ATEEM (Holt & Roff 2004) and the

practice-based educational environment measure (Mulrooney

2005) were developed for use in the surgical/operating theatre

learning setting, anaesthetic theatre learning setting and

practice-based component in general practice (GP) training,

respectively. The VA Learners’ Perceptions Survey was

designed for residents from different specialties within the

United States VA clinical training setting (Keitz et al. 2003). In

the process of deciding the most suitable instrument for use in

postgraduate medical education, it was appropriate to exclude

those instruments. The decision was not based on the quality

of those instruments, since most of them demonstrated good

validity and reliability, but rather because of the specific

content of the instrument, which limited their use in a more

general postgraduate education setting.

The content validity of the LEA was not established,

although it showed fairly high internal consistency (Roth

et al. 2006). The questionnaire from Rotem et al. (1995) and

the PHEEM established content validity through clear descrip-

tion of the development process of the instruments (Roff et al.

2005). The strengths of the DREEM were the high internal

consistency and concurrent validity. However, the content and

construct validities were not established since the factor

analysis demonstrated that some modifications were necessary

for the instrument to be applied in postgraduate setting

(Bassaw et al. 2003; de Oliveira Filho & Schonhorst 2005;

de Oliveira Filho et al. 2005a, b).

The PHEEM consisted of 40 items divided into three

subscales (perceptions of role autonomy, perceptions of

teaching and perceptions of social support). The reliability of

the PHEEM was better compared to that of the Rotem, Godwin

and Du questionnaire. It has also been shown that by using

PHEEM, reliable results can be obtained with feasible sample

size. In addition, the PHEEM had been administered to several

different sample groups, such as senior house officers and

specialist registrars from different specialties, and demon-

strated almost similar reliability coefficients in those groups

(Jayashree 2004; Roff et al. 2005; Aspegren et al. 2007; Boor

et al. 2007; Clapham et al. 2007). Some researches have

conducted factor analysis on PHEEM and they demonstrated

different results regarding the factors, which the instrument

measures (Aspegren et al. 2007; Boor et al. 2007; Clapham

et al. 2007). The construct validity of the Rotem, Godwin and

Du questionnaire was not clearly established. Therefore, the

PHEEM is likely to be the most suitable instrument for use in

postgraduate medical education because of its content validity,

high reliability and also its ability to be used in different

postgraduate settings.

Nursing. Most of the identified instruments were used to

assess the clinical learning environment in nursing education

setting. A summary of the psychometric qualities of the

instruments’ is presented in Table 4, available at www.

medicalteacher.org. All instruments, except CUCEI (Fisher &

Parkinson 1998), provided moderate to high reliability coef-

ficients. Several instruments, such as the CPCLES (Letizia &

Jennrich 1998), the questionnaire from Hart and Rotem (1995),

and the DREEM (Pimparyon et al. 2000; Al-Sketty 2003;

BEME Rapid Review

951

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Maa

stri

cht o

n 06

/18/

14Fo

r pe

rson

al u

se o

nly.

O’Brien et al. 2008) demonstrated some weaknesses in regard

to the construct validity of the instruments. Almost all

instruments have demonstrated a high degree of content

validity. The PREQ covered several aspects of an educational

environment (supervision, intellectual climate, clarity, infra-

structure, skills development and thesis examination process).

However, according to the validity analysis, there are some

subscales or statements in the instrument that were too specific

for a particular nursing education setting (Drennan 2008),

which then limit its use at different nursing education settings.

The content and construct validities of the CLES have been

established. It was administered to the groups of nursing

students from different countries (Finland and United

Kingdom) and it has demonstrated marginal to high reliability

coefficient (internal consistency) in those studies (Saarikoski &

Leino-Kilpi 1999, 2002; Saarikoski et al. 2002). The CLEI had

also been administered in several contexts of nursing educa-

tion. However, the internal consistency was fairly poor

compared to that of other instruments (Chan 2001a, b, 2002,

2003; Ip & Chan 2005; Henderson et al. 2006; Midgley 2006;

Chan & Ip 2007). The reliability coefficients of the CLE Scale

(Dunn & Burnett 1995; Dunn & Hansford 1997) were lower

than those of the CLES, although both the instruments’

construct validity has been confirmed. Therefore, based on

the available information and analysis, the CLES is likely to be

the most suitable instrument to assess nursing students’

perceptions of their clinical learning environment.

Dentistry. There were five instruments for dental educational

environment measurement identified in this study. The validity

and reliability of those instruments varied and are summarized

in Table 5, available at www.medicalteacher.org.

There was only one instrument that was originally devel-

oped for dental educational environment (Gerzina et al. 2005).

The others were modifications or identical version of instru-

ments, which were originally designed to be used in medical

school. The content validity of those instruments was ques-

tionable since there were no processes of validating them

before they were applied in dental education settings,

although the original versions may have proven to be valid.

In addition to that, two instruments (LES and DSLES) were

developed from the MSLES. It was an instrument first devel-

oped in 1970s and it might not encompass current trends or

changes in educational practice nowadays. The DSLES was

more identical to the MSLES than the LES, because the LES was

a shorter version of the MSLES whereas the changes in the

DSLES were only minor.

The LES (Stewart et al. 2006) demonstrated the highest

overall reliability coefficient (Cronbach’s alpha of 0.97). The

reliability of the ClinEd IQ (Henzi et al. 2006), the DREEM

(Zamzuri et al. 2004) and the questionnaire from Gerzina et al.

(2005) were not tested. The DSLES also demonstrated good

reliability (Cronbach’s alpha of 0.91), which was comparable

to those of the MSLES (Henzi et al. 2005).

Based on the information provided in the studies and the

analysis conducted, the DSLES is likely to be the most suitable

instrument for measuring educational environment in dental

education setting. It demonstrated good reliability and since

the DSLES was more similar to MSLES than the LES, it was

more likely to have better content validity.

Conclusion

The systematic search yielded 178 studies, which were

considered to be eligible for further review. The pre-specified

criteria were applied to those studies and finally 79 studies,

which were the original/primary studies measuring the overall

educational/learning environment in health professions edu-

cation using specific instruments, were included. A substantial

number of educational environment measurement instruments

(31 instruments) were extracted from the included studies. The

use of a wider range of databases with more elaborated search

strategies will increase the comprehensiveness of the system-

atic review.

The DREEM, PHEEM, CLES and DSLES are likely to be the

most suitable instruments for undergraduate medicine, post-

graduate medicine, nursing and dental education, respectively.

The content validity was established through elaborate

description of the development process of the instruments.

Some instruments have added values with the establishment of

their construct validity. These educational environment mea-

surement instruments also demonstrated consistency through-

out their applications in different contexts/settings.

Further analysis will be useful to explore the ability of each

instrument as a predictor variable for a particular educational

achievement. Furthermore, a study on students’ perceptions of

the educational environment should become a part of good

educational practice of an institution, as a suitable, valid and

reliable instrument for each educational context is available.

Acknowledgements

The authors thank Sue Roff and Sean McAleer for their

comments and support in completing the project and finalizing

the manuscript. This work is partially supported by grants of

the National Commission for Scientific and Technological

Research (CONICYT), FONDECYT no. 11004336 to A.R.

Declaration of interest: The authors report no conflicts of

interest. The authors alone are responsible for the content and

writing of the article.

Notes on contributors

DIANTHA SOEMANTRI, MD, MmedEd, is a lecturer in medical education in

the Department of Medical Education, Faculty of Medicine, Universitas

Indonesia.

CRISTIAN HERRERA is member of the Evidence Based Medicine Unit and

the Health Policy and Systems Research Unit from the School of Medicine,

Pontificia Universidad Catolica de Chile.

ARNOLDO RIQUELME, MD, MmedEd, is an undergraduate and postgrad-

uate clinical tutor in Internal Medicine and consultant in the Department of

Gastroenterology, Pontificia Universidad Catolica de Chile School of

Medicine, Chile.

References

The references for this article can be viewed at www.medicalteacher.org.

D. Soemantri et al.

952

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Maa

stri

cht o

n 06

/18/

14Fo

r pe

rson

al u

se o

nly.