This article was downloaded by: [Colorado College]On: 28 October 2014, At: 17:16Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Medical Reference ServicesQuarterlyPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/wmrs20

Evaluation of a HospitalMedical Library Class for NICUNursesMisa Mi MA, MLIS, AHIP aa Medical Library, Children's Hospital of Michigan ,3901 Beaubien Boulevard, Detroit, MI, 48201, USAPublished online: 25 Sep 2008.

To cite this article: Misa Mi MA, MLIS, AHIP (2006) Evaluation of a Hospital MedicalLibrary Class for NICU Nurses, Medical Reference Services Quarterly, 25:4, 21-35

To link to this article: http://dx.doi.org/10.1300/J115v25n04_03

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,

sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

Evaluation of a Hospital Medical LibraryClass for NICU Nurses

Misa Mi

ABSTRACT. A library class was designed and offered to new nursesfrom the Neonatal Intensive Care Unit at the Children’s Hospital ofMichigan between 2003 and 2005. The class was intended to increasetheir knowledge of quality health information resources and to assistthem with their smooth transition to a new health care organization. Thegoal of the library training class was to develop the nurses’ awarenessand knowledge of the library services and online resources on the orga-nization Intranet and to improve their skills in finding reliable informa-tion related to patient care, patient parent education, and research. Anevaluation study was conducted to assess the effectiveness of the libraryclass. Although the findings demonstrated strengths of the library class,they also revealed some areas for improvement. The data gathered re-sulted in a number of recommendations regarding library instructiondesign and evaluation. doi:10.1300/J115v25n04_03 [Article copies availablefor a fee from The Haworth Document Delivery Service: 1-800-HAWORTH.E-mail address: <docdelivery@ haworthpress.com> Website: <http://www.HaworthPress.com> © 2006 by The Haworth Press, Inc. All rights reserved.]

KEYWORDS. Library training, library class, instructional design, evalu-ation, summative evaluation, nurses, neonatal intensive care unit

INTRODUCTION

Nurses working in a neonatal intensive care unit (NICU) in any hospital set-ting deal with high-risk infants who are often in critical condition and have nu-

Misa Mi, MA, MLIS, AHIP (mmi@dmc.org) is Librarian, Medical Library, Chil-dren’s Hospital of Michigan, 3901 Beaubien Boulevard, Detroit, MI 48201.

Medical Reference Services Quarterly, Vol. 25(4), Winter 2006Available online at http://mrsq.haworthpress.com

2006 by The Haworth Press, Inc. All rights reserved.doi:10.1300/J115v25n04_03 21

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

merous acute or chronic medical problems. These nurses not only need to takecare of these patients, but also support and educate parents and families abouttheir child’s condition and treatment/procedure. The nurses’ unique role anddaily tasks have made it necessary for them to develop skills in finding and us-ing resources to meet their information needs on the patient floors. To developthe nurses’ information literacy skills and help them become familiar with awide variety of library services and resources, the librarian at the Medical Li-brary of the Library Services Department at the Children’s Hospital of Michi-gan, developed a class targeted to the NICU nurses at the hospital. Between2003 and 2005, as many as 32 nurses participated in the library training.

A summative evaluation can be conducted to help make decisions aboutprogram adoption, continuation, or expansion. The evaluation can assist edu-cators and the target audience in making judgments about a program’s overallworth or merit in relation to important criteria.1 A summative evaluation wasconducted to assess the effectiveness of the library class. The purposes of theevaluation were to achieve the following:

• Evaluate the NICU nurses’ performance of information searching skills.• Provide information to nurse managers and nurse educators who will

need to know the value of the training to decide whether the library train-ing should continue for new nurses and be integrated into the orientationfor new NICU nurses.

• Provide the librarian at the medical library with preliminary data on theeffectiveness of the library class.

• Show how and where to improve efforts in instructional design of the li-brary class targeted to NICU nurses in the hospital.

DESCRIPTION OF THE LIBRARY CLASS

The library class was designed for newly hired or beginning NICU nurses.A librarian at the medical library served as a subject matter expert, designingand developing a three-hour class for the nurses. She was the instructor for theclass as well. The instruction content consisted of a combination of a presenta-tion, demonstration, and exercises. The prerequisites required for the nurseswere their ability to use Internet Explorer and to have access to a hospital net-worked computer. Class sizes ranged from three to six nurses. The instructionwas conducted in a computer lab equipped with nine PC computers. Researchon assessment of nurses’ information needs suggested that nurses were partic-ularly interested in patient education, drug and disease information, diagnostic

22 MEDICAL REFERENCE SERVICES QUARTERLY

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

test information, and literature searching. The instruction content was createdwith emphasis on these areas of interest (see Table 1).

To engage the nurses in learning and make learning content applicable totheir clinical setting, a pre-training assessment was performed to seek out whatthey expected to learn from the class. The assessment was intended to ensurethat the nurses’ learning objectives or interests were considered and incorpo-rated into the instructional objectives that the course content was designed toachieve.

In the first section of the training class, the instructor explained to thenurses what library services were available to them and how they could alsorequest the services on the library Intranet Web site. It was also important tolet the nurses know they had all the library privileges as physicians since somenurses thought the medical library was just for physicians. Following the in-troduction, the librarian led a five-minute tour of the physical location of thelibrary and showed them the print journal and book collections, computers,and copy machines.

The second section focused on the library Web site on the Intranet and itsonline catalog. The nurses learned how to access the library Web site locally

Misa Mi 23

TABLE 1. The Key Components of the Library Training Class

1. Introduction to library services and resourcesInterlibrary loanLiterature searchCirculationLATCH (literature attached to patient’s chart)Library training/instructionLibrary tour

2. Library intranet web site and online catalog3. Patient education

Family resource libraryLibrary digital collection of patient parent education materialsCareNotesRecommended pediatric Web resourcesMedlinePlus

4. Online full-text medical and drug databasesSTAT!RefMICROMEDEX

5. Bibliographical databasesCINAHLPubMed

6. Online journals

7. Practice questions

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

on the patient floors and remotely from home or off-site clinics. The instructordemonstrated how to find links to important nursing resources such as CINAHLand patient education resources. Most of the nurses were graduates of severalnursing schools in Michigan universities. All university libraries use differentsystems for their online catalog, which results in different search interfaces.Therefore, the section was also intended to familiarize the nurses with the li-brary catalog so that they could easily locate information on books and journalholdings owned by the library.

The third section concentrated on patient education resources. Patient educa-tion is an integral part of the nurses’ daily work, especially in the pediatric healthcare setting where nurses need to work with parents and families and educatethem about their child’s disease or condition. As NICU nurses, they workclosely with parents and families who have many different needs such as infor-mational, emotional and physical needs, while their sick babies are hospitalized.The nurses expressed a strong interest in finding patient education materialswhen filling out the pre-training assessment questionnaire. The instructor madesure there was enough time in this section to cover important resources. The pre-sentation started with a brief introduction to a family resource library, part of theLibrary Services Department. The library is a place serving patients, parents,and families, who are welcome to ask about health information and borrowitems from its children’s health and literature collections. The medical librarymaintains a popular Web-based collection of over 400 full- text patient parenteducation handouts written in house, which are a result of a joint project be-tween the librarians and pediatric nurse educators in the hospital. The nurseslearned how to locate a specific handout by category, title, or nurse unit. A list ofannotated pediatric Web resources compiled by the librarian was also presentedto the nurses. Another source of patient education materials was CareNotesthrough MICROMEDEX, available on the Intranet. The nurses were shown itscontent list and types of patient education materials available in the database. Fi-nally, the nurses were directed to MedlinePlus from the National Library ofMedicine, with its authoritative and up-to-date health information.

In the fourth section, the instructor presented STAT!Ref and MICRO-MEDEX, which were new to all nurses who participated in the training. Nursingstudents in most nursing schools are rarely familiar with these since the data-bases are only available in some academic health sciences and hospital medicallibraries. Research suggested that books were preferred resources for patientcare among clinical nurses because books provide immediate access to con-cise information.2 The instructor highlighted nursing and drug book titlesavailable in STAT!Ref that the nurses could access 24/7 on the patient floorsand search for information on therapies, diagnostic tests, and drugs. MICRO-MEDEX provides comprehensive drug, acute care, and toxicology informa-

24 MEDICAL REFERENCE SERVICES QUARTERLY

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

tion. In addition to showing the nurses different ways to searching the data-base, the instructor demonstrated how to use several tools in MICROMEDEX:

• PDR;• Drug Interactions information which allows users to enter a patient’s

prescribed medications one at a time for a list of possible interactions;• Dosing Tools including calculators, nomograms, and charts and tables of

pertinent medical, pharmaceutical, or toxicologic facts not easily found inone place or via other sources;

• IV Compatibility that provides test results for two-drug combinations inthree different infusion solutions.

Many newly hired nurses were familiar with CINAHL, since they had touse it for their research projects at school. Some accessed the database throughOCLC FirstSearch, others through Ovid. In the fifth section, the instructorspent a few minutes in showing the nurses where to access CINAHL on the pa-tient floors or remotely from home. In Ovid CINAHL, the features such as“Mapping to Subject Heading,” “Explode,” and “Focus” were briefly re-viewed. The instructor had noticed through her experience working with nurs-ing students that most did not know they could expand or focus their searcheswhen using Ovid CINAHL. This section also introduced the nurses to PubMedas a resource for information on patient care and nursing research.

The sixth section directed the nurses to the Web page for the library collec-tion of online journals searchable by subject, title, or category. The onlinejournal collection accessed through EBSCO provides an easy and convenientway to find full-text articles. Many nurses worked on night shifts and seldomhad a chance to visit the library during regular business hours. Access to full-text articles or textbooks on the networked computers could make it easy forthem to find information related to patient care and increase their use of the on-line resources.

The last part of the training was a hands-on practice on finding answers to20 questions. The nurses had 30 minutes to work in pairs to search for answersto the questions by using the online resources introduced in the class. The pur-pose of the hands-on practice was to give the nurses an opportunity to practiceand reinforce their learning with their peers.

The library class was a one-time training session for the nurses. To helpthem review and retain the knowledge and skills learned in the instruction, apackage of handouts was prepared beforehand and handed out to the nursesduring the class. The package contained the following information:

Misa Mi 25

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

1. User guides/manuals for searching CINAHL, PubMed, STAT!Ref,MICROMEDEX

2. Instruction sheets for searching the library catalog and using theLATCH service

3. Library service request forms4. Library services promotion brochure5. Practice questions6. Pre-training assessment7. Post-training evaluation sheet.

The instruction was delivered by the librarian from the medical library withexpertise and experience in teaching online searching and knowledge of medi-cal and nursing resources. The librarian also conducted the evaluation study.

When new nurses started working in the NICU department, they alwayswent through intensive orientation. Two nurse educators who were in chargeof NICU new nurse orientation decided to integrate the library training classinto the whole orientation schedule. The nurse educators served as liaisons be-tween the new nurses and the library. They identified the nurses who needed orwere interested in the library training, scheduled the training session with thelibrarian, and provided the librarian with information such as their name andtime slot for their library training during their orientation weeks.

EVALUATION METHODS

The evaluation was designed as a summative evaluation in an attempt to an-swer the following four questions:

1. Did the library class develop nurses’ awareness of the library healthinformation resources?

2. Was the training class effective?3. Did the training class improve nurses’ skills in retrieving health infor-

mation?4. Were the nurses satisfied with the library training class?

Participants

Twenty-eight nurses from the neonatal intensive care unit, all female,enrolled in the library training class during various times between 2003 and2005. Most of them were new nurses who had worked in the hospital for onemonth to one year. Some of them worked on day shift and others on night shift.All the nurses had access to their organization e-mail system and to the hospi-

26 MEDICAL REFERENCE SERVICES QUARTERLY

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

tal networked computers at nursing units or on the patient floors. From anynetworked computer, they could access patient records on the Intranet andsearch for information on the Internet. All of them except one had indicatedthey had used Internet Explorer. The pre-training assessment showed thatmost of them possessed the required prerequisite skills for the library class.Only six nurses indicated they used the library weekly while most of themrarely used the library. Like physicians, nurses are provided with 24-hourbadge access to the library resources and facilities. When the library is notstaffed, nurses can still enter the library to use the resources and facilities suchas computers and photocopy machines.

Settings

A computer lab (not part of the medical library) located next door to themedical library was utilized for the library class. There were nine PC comput-ers with Internet access. When the class size was over six participants, a dataprojector was used to assist the delivery of the instruction.

Instrumentation

Pre-Training Assessment. A pre-training assessment consisting of eightquestions (four open-ended and four closed-ended questions) was conductedto determine the nurses’ library use pattern, access to networked computers,knowledge of the library online resources, and expectation of the library class.Since the nurses were new to the hospital, the librarian instructor did not havea chance to get to know them ahead of the training or during the process of de-signing and developing the training materials. The assessment could providethe instructor with some baseline information on the nurses’ characteristics asadult learners. It could serve as a tool to elicit the nurses’ felt needs about whatthey desired or wanted to learn through the library class. It was hoped that theshort assessment questionnaire would yield some information on the nurses’prior knowledge of online resources and prerequisite skills in using theInternet and online databases. The information could also be used to determinethe level at which topics were introduced and the depth of the topics covered inthe training.

Post-Training Evaluation. A post-training evaluation questionnaire wasadministered to the nurses upon the completion of the class. The questionnaireconsisted of eight questions, five of which were modified Likert Scale ques-tions. The five modified Likert Scale questions were intended to elicit thenurses’ reactions and opinions on the effectiveness of the instruction and train-ing handouts, and their level of satisfaction with the training and librarian

Misa Mi 27

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

instructor. One question was included to collect the participants’ extra com-ments about the library training class, and another solicited their suggestions onfuture workshops or presentations.

Observation. The nurses spent 30 minutes on a hands-on practice, findinganswers to 20 questions, by using the online library resources and databases.The questions were created as an activity to enhance their learning as well as ameans to assess the nurses’ acquired knowledge and skills. While they workedon the questions individually or in pairs, the instructor/evaluator roamedaround the lab and observed the nurses while they were working on the ques-tions. The instructor observed their information-seeking behaviors during theunstructured observation and provided assistance or feedback as needed. Shealso took note of the number of service requests that the NICU nursessubmitted to the library after the training.

RESULTS

Pre-Training Assessment

A total of 22 pre-training assessment sheets were collected from the nurseswho took the library class. All the nurses who participated in the training hadaccess to the networked computers and had experience using Internet Ex-plorer. However, most of them had limited use of the library. Only 6 out of 22nurses indicated that they had used the library catalog.

When the nurses were asked where they would search for information oncongenital heart defects, it was not surprising to note that Yahoo! and theInternet were at the top of the list. Only three nurses used the recommended re-sources to answer the question: two responses were CINAHL and one re-sponse was MedlinePlus. Obviously, most of the nurses were not aware of thewide variety of reliable resources available on the library Web site, and theyquickly chose Internet search engines with which they were familiar and couldeasily locate.

However, when asked what resources or databases they had used, sevennurses wrote down CINAHL, three MEDLINE, and three FirstSearch. Onementioned InfoTrac and Winspirs. These nurses had just graduated from col-lege, which may explain their familiarity with CINAHL. The information in-dicated that some new nurses had experience using CINAHL and several otheronline databases. Yet, most respondents listed the Internet search engines asthe resources they had used. These search engines included Google, Yahoo!,MSN, or AOL. Two indicated they had not used any resource.

28 MEDICAL REFERENCE SERVICES QUARTERLY

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

The pre-training assessment also attempted to explore what expectationsthe nurses had about the library class and what needs or desires the nurses had.After their comments were examined, they were grouped into three categoriesof needs or expectations: access to information, knowledge of resources andservices, and information-seeking skills (see Appendix 1). These needs andexpectations were integrated into the instruction.

Post-Training Evaluation Survey

Twenty-eight post-training evaluation questionnaires were completed andcollected for the evaluation study. The questionnaire consisted of five modi-fied Likert scale questions and a section seeking out any comments and sug-gestions from the nurses. Three questions asked the nurses’ opinions about theeffectiveness of the instruction and handouts. The rating scale ranged from 5(Strongly agree) to 1 (Strongly disagree). The scale used 4, 3, and 2 as the re-sponses of Agree, Neutral, and Disagree. Out of 28 nurses (46.4%) evaluated,13 strongly agreed that the topic was interesting. Half of them (50%) stronglyagreed that the information was clearly presented. Over one-half of the nurses(60%) agreed that the training and user’s guide would help them perform theirliterature search. For Questions 2 and 4, two nurses gave a “Neutral” response,while one response was “Disagree” for the two questions (see Figure 1).

The rating scale for Question 5 ranged from 5 (Very satisfied) to 1 (Not sat-isfied). The scale used 4, 3, and 2 as the responses of Satisfied, Neutral, andFair. Sixteen (57%) respondents indicated that they were very satisfied withthe training class, and over 28% were satisfied with the training (see Figure 2).

Misa Mi 29

07.1

39.346.4

0 0

10.7

39.3

07.1

28.6

3.6

50

60.7

3.60

10

20

30

40

50

60

70

1 2 3 4 5

Strongly Disagree Strongly Agree

Per

cent

age

ofR

espo

nden

ts

2. Was the topic interesting to you?

3. Was the information clearly presented?

4. Will the training and user’s guide help perform your literature search?

FIGURE 1. Questions 2, 3, and 4

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

Question 6 was intended to evaluate the instructor. The rating scale rangedfrom 5 (Excellent) to 1 (Poor). The scale used 4, 3, and 2 as the responses ofVery good, Good, and Fair. Over 60% (n = 17) of the respondents thought thatthe instructor was excellent, and more than 35% rated the instructor as verygood (see Figure 3).

Comments about the library class and suggestions for future workshops andclasses were collected as well. The positive feedback from the evaluationquestionnaire further indicated that the class was effective in achieving itsobjectives and that the nurses felt satisfied with the learning experience (see Ap-pendix 2). Some nurses commented that the training class was very informativeand helpful to them and that the instruction was easy to understand. Two nurseswere glad they learned how to access the library resources remotely from home.One nurse stated what most nurses felt about the training, “The presentationwas helpful to learn the basics of all databases and information that is availableto us when we help educate parents.”

On the negative side, a few comments reflected some problems that severalnurses experienced in the training. One commented that she didn’t understandthe purpose of this training. She felt that the training was too long and shouldbe related to how to obtain information for patients only. Another commentedthat the training was very repetitive and that she just needed to know where theresources were located. These comments may account for two low ratingsgiven to Questions 2 and 4. If the nurses felt they already knew the content,they may not feel motivated to learn. The other two nurses were not happyabout the computer lab, which was very cold during a particular training ses-sion. The physical condition in the learning environment could affect learningoutcome and learners’ satisfaction with instruction.

30 MEDICAL REFERENCE SERVICES QUARTERLY

0

10.8

28.6

57.1

3.6

0

10

20

30

40

50

60

1 2 3 4 5

Not Satisfied Very Satisfied

Per

cent

age

ofR

espo

nses

FIGURE 2. Question 5: What Was Your Overall Level of Satisfaction with theTraining Session?

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

A brief observation was conducted of the nurses in their post-training prac-tice. The evaluator noticed that most of the nurses started their search with thelibrary Intranet Web site instead of jumping to Google, Yahoo!, or otherInternet search engines. All of them could search the library catalog withoutany difficulty. For questions asking for a general overview on a disease, condi-tion, or treatment, the online resources they searched varied from one nurse toanother. Some used MedlinePlus, some STAT!Ref, a few others MICRO-MEDEX, but most of them could locate the answers by using the resources onthe library Web site. The instructor also observed that a few nurses were notsure about the best online resource to use to search for an answer. After two orthree trials, they located the answers to some questions in the exercises.

Library patrons at the hospital can submit their literature search or interli-brary loan requests online from the library Intranet Web site. The librariannoticed that some NICU nurses began to submit online literature search re-quests following their library training. During the training, the nurses were ad-vised to ask for professional assistance when they had difficulty in locatingmedical information related to their patient care and research. The increaseduse of the library services by the NICU nurses further demonstrated the effec-tiveness of the training–improving the nurses’ awareness and knowledge ofthe library services and resources.

RECOMMENDATIONS

The preliminary data collected resulted in the following recommendationsfor improvement in library class instruction design and evaluation.

• Nurses need a context to learn searching skills so that they can applytheir newly acquired knowledge and skills to their clinical setting. Activ-

Misa Mi 31

0 0

35.8

60.7

3.60

10

20

30

40

50

60

70

1 2 3 4 5

Poor Excellent

Re

sp

on

se

Ra

te

FIGURE 3. Question 6: How Do You Rate the Instructor?

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

ities or exercises should be designed and integrated into different sec-tions of the training class to motivate the nurses and help them applywhat they learn to patient care and education.

• Two nurses suggested that each section of the instruction content be givenin different classes. Therefore, the library training can be designed as a se-ries of one- or two-hour classes offered to nurses for a three- to four-dayperiod. Each day, one class can address one topic or one content area.

• Nurses who have some searching skills and prior knowledge of the li-brary services and resources should be encouraged to participate in atraining session focused only on resources unfamiliar to them (e.g.,STAT!Ref, MICROMEDEX, and Cochrane Library).

• The nurses in one training session complained about the cold tempera-ture in the computer lab. The low temperature could be distracting to thenurses. In future training session, the instructor should make sure that thelearning environment is conducive to learning and learning outcome.

• The library class should be provided to new nurses or nurses alreadyworking on the patient floors and can also be introduced to other depart-ments as well. Nursing managers from different departments should makethe library training class a part of new nurses’ orientation or nurses’in-service training in the hospital.

• Formative evaluation is an integral part of education and training. It helpsa course developer or instructor find out if she is achieving her goals andobjectives in the formative stages of her project, instead of waiting untilthe project is finished. In developing future library classes for nurses,formative evaluation should be conducted in the development stage ofthe class so that improvement and revisions can be made to improve theeffectiveness of the training for nurses.

• Each rating on the rating scale for the Likert scale questions in the post-training evaluation should be specified to avoid any confusion among therespondents and in the data analysis.

• Evaluation criteria such as instruction length, content coverage, and rel-evance should be incorporated into the questionnaire design to improvethe effectiveness of the instrument.

• Pre-testing of the evaluation instruments should be performed on a smallsample of the target audience. Revisions and improvements can be madebased on results of the pre-test to improve the validity and reliability ofan instrument.

• Interviews allow clarification and probing, and permit exploration anddiscovery.1 Structured interviews can be conducted with a small sampleof nurses to yield in-depth information on their perspectives and opin-ions about the effectiveness of a library training class.

32 MEDICAL REFERENCE SERVICES QUARTERLY

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

• Meta-evaluation can be conducted to evaluate how effective and reliablethe evaluation tools are in evaluating the success of the library trainingclass. Revisions can be made to the pre-training assessment and post-training questionnaire to improve their reliability and validity.

• Follow-up evaluation should be planned three or six months after thenurses receive the training to determine how the nurses can apply theirknowledge and information-seeking skills in finding health informationrelated to patient care and parent education.

CONCLUSION

An evaluation study was conducted to investigate the effectiveness of athree-hour library training class for the NICU nurses. The findings from theevaluative study suggested that, as a result of the training class, the nurses de-veloped better awareness of the library services and trusted health informationresources and improved their skills in searching information related to patientcare and patient education. The majority of the nurses were satisfied with thetraining and rated the instructor excellent.

Some limitations exist in the evaluation study. The class was instructed andevaluated by the librarian who designed the library training class. As an in-structional designer and instructor, she may be so close to the training or fa-miliar with the purpose and content of the class that she could not see theevaluation clearly. The closeness may prevent her from seeing solutions orchanges that an external evaluator might see more readily.

Received: January 11, 2006Revised: March 29, 2006Accepted: April 10, 2006

REFERENCES

1. Worthen, B.R.; Sanders, J.R.; and Fitzpatrick, J.L. Program Evaluation: Alter-native Approaches and Practical Guidelines. Boston: Allyn and Bacon, 2004.

2. Dee, C., and Stanley, E.E. “Information-Seeking Behavior of Nursing Studentsand Clinical Nurses: Implications for Health Sciences Librarians.” Journal of MedicalLibrary Association 93(April 2005): 213-22.

doi:10.1300/J115v25n04_03

Misa Mi 33

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

34 MEDICAL REFERENCE SERVICES QUARTERLY

APPENDIX 1

Pre-Training Assessment Comments

Access to Information

• To learn how to get journal articles and other information from the hospital li-brary, so I can present a paper or project for my co-workers. Also to get informa-tion for patients and parents.

• Learn how to access medical information easier.• How to access DMC sites and library information.• A better understanding of how to access information using a DMC networked

computer.

Knowledge of Resources and Services

• Better understanding of what kind of resources are on the Internet and how to ac-cess them.

• To learn about new databases.• What materials are available in the library.• To learn what other services are available from computers and library.• Everything about DMC page.• Expanded areas to search for subjects.• Research information.• Basic computer information regarding work.• To see what is available to use.

Information-Seeking Skills

• Learn how to use the DMC online library catalog.• How to do an online search.• Able to find “good” information from creditable resources online.• How to use resources.• How to research.• To be able to research through the computer.• Information on quick ways to find medical information.

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4

Misa Mi 35

APPENDIX 2

Comments Generated from the Post-Evaluation Questionnaire

Positive Comments

• The presentation was helpful to learn the basics of all databases andinformation that is available to us when we help educate parents.

• Great job! Very easy to understand. Thank you.• We did not have a lot of time but the information was helpful to me

and I think I can find information now.• Short and to the point.• Very informative. Handouts will be helpful to look information

up at home.• Good highlights on system and how to use even at home.

The librarian made it clear she was available for future questions.• Very informative. I wish I’d had this type of instruction when

I was in school.• Excellent for basic information.• Very helpful presentation.• The librarian is very knowledgeable.

Negative Comments

• I didn’t understand the purpose of this session. I am not in schoolanymore where I would be doing a lot of research. I feel it should berelated to how to obtain information for patients only. Also, the time wastoo long.

• The room is very cold.• I understand how to use the databases. I just needed to know where

they were. This was very repetitive.• Already received training on finding patient education handouts

and researching topics from multiple years of university.STAT!Ref and MICROMEDEX were only new materials–too longfor already knowing materials.

Dow

nloa

ded

by [

Col

orad

o C

olle

ge]

at 1

7:16

28

Oct

ober

201

4