cpd framework

8/3/2019 CPD Framework

1/13

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 4 ( 2 0 0 9 ) 2638

j o u r n a l h o m e p a g e : w w w . i n t l . e l s e v i e r h e a l t h . c o m / j o u r n a l s / c m p b

Development of a web database portfolio system with PACS

connectivity for undergraduate health education and

continuing professional development

Curtise K.C. Ng a,, Peter White a,1, Janice C. McKay b,2

a Department of Health Technology & Informatics, The Hong Kong Polytechnic University,

Hung Hom, Kowloon, Hong Kongb Discipline of Medical Imaging, Department of Imaging and Applied Physics,

Curtin University of Technology, PO Box U1987, Perth, WA 6845, Australia

a r t i c l e i n f o

Article history:

Received 11 April 2007

Received in revised form

31 October 2007

Accepted 6 October 2008

Keywords:

Electronic portfolio

Picture archiving and

communication system

Education

Continuing professional

development

Database

Reflection

a b s t r a c t

Increasingly, the use of webdatabaseportfolio systems is notedin medical andhealth educa-

tion, and for continuing professional development (CPD). However, the functions of existing

systems are not always aligned with the corresponding pedagogy and hence reflection is

often lost. This paper presents the development of a tailored web database portfolio system

with Picture Archiving and Communication System (PACS) connectivity, which is based on

the portfolio pedagogy.

Following a pre-determined portfolio framework, a system model with the components

of web, database and mail servers, server side scripts, and a Query/Retrieve (Q/R) broker for

conversion between Hypertext Transfer Protocol (HTTP) requests and Q/R service class of

Digital Imaging and Communication in Medicine (DICOM) standard,is proposed. The system

was piloted with seventy-seven volunteers.

A tailored web database portfolio system (http://radep.hti.polyu.edu.hk) was developed.

Technological arrangements for reinforcing portfolio pedagogy include popup windows

(reminders) with guidelines and probing questions of collect, select and reflect on

evidence of development/experience, limitation in the number of files (evidence) to be

uploaded,the EvidenceInsertion functionality to linkthe individualuploaded artifacts with

reflective writing, capability to accommodate diversity of contents and convenient inter-

faces for reviewing portfolios and communication. Evidence to date suggests the system

supports users to build their portfolios with sound hypertext reflection under a facilita-

tors guidance, and with reviewers to monitor students progress providing feedback and

comments online in a programme-wide situation.

2008 Elsevier Ireland Ltd. All rights reserved.

Corresponding author. Tel.: +852 3400 8587; fax: +852 2362 4365.E-mail addresses: curtise [email protected], curtise [email protected] (C.K.C. Ng), [email protected] (P. White),

[email protected] (J.C. McKay).1 Tel.: +852 3400 8568; fax: +852 2362 4365.2 Tel.: +61 8 9266 7151; fax: +61 8 9266 4344.

0169-2607/$ see front matter 2008 Elsevier Ireland Ltd. All rights reserved.

doi:10.1016/j.cmpb.2008.10.004
http://radep.hti.polyu.edu.hk/mailto:[email protected]:[email protected]:[email protected]:[email protected]://dx.doi.org/10.1016/j.cmpb.2008.10.004http://dx.doi.org/10.1016/j.cmpb.2008.10.004mailto:[email protected]:[email protected]:[email protected]:[email protected]://radep.hti.polyu.edu.hk/


2/13

c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 4 ( 2 0 0 9 ) 2638 27

1. Introduction

Increasingly, the use of a web database portfolio system is

noted in areas of medical and health education, and for con-

tinuing professional development (CPD) [13]. Within health

care, for either taught programmes or CPD purposes, both

academic and professional development should be noted,and students or practitioners should be encouraged to make

the connection between the two, to develop the ability to

adapt and adjust learning informed by experience and prac-

tice. A portfolio is an instrument which can support both

the outcomes-based educational approach and requirements

for meeting competence in health care education and pro-

fessional accreditation processes. The purpose of both is

usually to foster reflection on previous learning and prac-

tice experiences, leading to applied learning and continuous

development in a cyclic manner [47]. Electronic portfolios

create increased opportunities, when compared with the

traditional paper-based formats, to support and facilitate

professional development from pre-registration education tolater CPD activities. Based on the enormous capability of stor-

age and management, they provide a link for the illustration

of professional development to different parties such as aca-

demic institutions, registering bodies and employers [811].

2. Background

2.1. Analysis of related work

Web database portfolio systems create the possibility to

promote a large scale portfolio practice and to provide learn-

ing and assessment opportunities, and tracking capability

of development for pre-registration education and CPD. Theexistingcommercial webdatabase portfoliosystems,however,

do not fit with these purposes probably due to the conse-

quences of the electronic portfolio boom [9,12]. The great

financialpotentialhas attracted corporate and academicinsti-

tutions such as McGraw-Hill and Wadsworth, and prestigious

universities such as Stanford University and the University of

California, Los Angeles (UCLA) to share in this huge market.

However, financial profitability depends on efficiency lead-

ing to distortion of the portfolio pedagogy. The functions of

these systems are not aligned with the pedagogy, i.e. the

ability to collect, select and reflect on evidence of develop-

ment/experience. Instead, there is a branching effect to pro-

vide functions other than those normally required of a port-folio, including online resumes, course management tools,

advising portfoliotools, institutionalportalsand content man-

agement. From a pedagogical perspective, the widening foci

create a threatto theeffectiveness of theportfolio pedagogyby

users using the related functions provided rather than build-

ing their own portfolios. Together with the standardization of

the portfolio building process to increase efficiency, the core

component of the portfolio, i.e. reflection, is lost [9].

2.2. Prior work leading to current design decisions

An emphasis on reflective learning and professional devel-

opment can be captured using electronic portfolio systems.

What is important is that the system can facilitate authors to

meet the content requirements of a portfolio rather than the

technical process of portfoliobuilding. Thispaper presentsthe

development of a tailored webdatabase portfoliosystem, built

for use by radiographers, which is strongly based on the port-

folio pedagogy, so as to capture their reflective learning and

professional development. Flexible tools are granted to users

as authors of the portfolio rather than as content providers oruploaders of artifacts, with some standardization to ensure

efficiency. Picture Archiving and Communication System

(PACS) connectivityis alsoinjectedinto the system to providea

means of facilitatingCPD andevidence of life long learning for

radiographers by capturing, for example, their workplace out-

comes, such as medical images produced. Since CPD is devel-

oping among all health professions, it could also be extended

to other professional areas so as to provide evidence of prac-

titioners competences throughout their professional lives.

3. Design considerations

To develop a web database portfolio system based on the

portfolio pedagogy, a portfolio framework was determined in

advance. The following discussion describes the setting up of

a portfolio system used on a voluntary basis by students on

an undergraduate radiography programme. Students expect

to receive strong support in their professional development,

including progress towards and ability to meet required grad-

uating professional competences. A reflection workshop to

teach the students how to collect and select relevant artifacts

was arranged after appropriate standards of progress were

delivered at corresponding stages. Students prepared evi-

dence to demonstrate meeting these stages from, for example,

assessment instruments. This was placed into the portfo-lio along with their own reflective writing to link up the

individual pieces of evidence and to illustrate the level of

competence attainment relevant to the standard. This incor-

porates the mechanism of the portfolio in tying up evidence

from multi-dimensional assessment strategies [13]. Through-

out the academic year, a facilitator was available to provide

additional support and guidance to students who had diffi-

culties with the portfolio pedagogy, such as assisting them to

recognize their own development. Evaluation of the portfo-

lio by faculty was conducted by reading the reflective writing

and following the links to the corresponding individual arti-

facts. They provided ratings of attainment of each competence

statement noted. Faculty then provided feedback to eachparticipant to inform them of their progress of competence

developmentalong with suggested remedieswhere necessary.

Through this process, faculty obtained a more in-depth view

of each students learning and development progress for pro-

gramme evaluation. This processwas repeated over two years.

As a formal component of a programme, this practice can

be repeated throughout the students study until graduation

as a means of monitoring their progress towards, and ability

to meet, graduating professional competences. Although the

portfolio was developed based on an undergraduate radiog-

raphy programme, the framework is suitable for other health

careprogrammesor CPD purposesby substitutingcorrespond-

ing competences for different professions [14].


3/13

28 c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 9 4 ( 2 0 0 9 ) 2 6 3 8

Fig. 1 A model of a web database portfolio system with

PACS connectivity.

4. System description

4.1. Model of web database portfolio system with

PACS connectivity

A model of a web database portfolio systemis proposed (Fig. 1)

taking into consideration the situation of the portfolio frame-

work and the possibility of providing a means of facilitating

CPD and providing evidence of life long learning by captur-

ing artifacts such as medical images produced. Web clients

such as undergraduate students, clinical practitioners, facili-

tators and reviewers can use browsers to communicate with

a web server through internet, to access the system. The

web server runs appropriate Active Server Pages (ASP) scripts

(ASP-VBScript) to provide requested services such as user

authentication, portfolio building and viewing, feedback and

grading,internalmessagecommunication, memorandum andchangesof userpreferences. The ASP scriptsprovidefunctions

of file upload and connectivity with mail, database and PACS

servers apart fromgeneralweb pagerendering. The webserver

in this study is a Dell PowerEdge 2850 using Internet Informa-

tion Server (IIS) 6.0 from the Microsoft Server 2003. The mail

server uses the default Simple Mail Transfer Protocol (SMTP)

virtual server inside the IIS 6.0 to send out automated Elec-

tronic Mail (e-mail), so as to alert users. The database server

is another Dell PowerEdge 2850 machine running Microsoft

SQL Server 2005 in a Microsoft Server 2003 platform. This is

responsible for archiving and retrieval of data for system con-

tents rendering and users input through Object Linking and

Embedding Database (OLEDB) connection. Capturing of med-ical images produced by clinical radiographers is achieved by

the Query/Retrieve (Q/R) service class of Digital Imaging and

Communication in Medicine (DICOM) standard. The Q/R bro-

ker converts Hypertext Transfer Protocol (HTTP) requestsfrom

clients to Q/R service class including C-Find, C-Move and C-

Store for a PACS server, and retrieved DICOM images/data to

HTTP responses for browser display and archiving in a web

database [15].

4.2. Implementation and evaluation

The system was piloted with seventy-seven volunteers

(seventy-four authors and three reviewers) from an under-

graduate radiography programme. There were thirty-five

students (authors) from the first year group, twenty from

the second year and nineteen from the final year groups.

An evaluation questionnaire of the system called the ePort-

folio questionnaire was delivered to them after they had

experienced two cycles of electronic portfolio practice. This

aimed to gather users views on the electronic portfolio

practice from their experience of using it, including con-ceptual and technical aspects. Feedback then informed the

revision/modification/improvement of the concept and sys-

tem. For the questionnaire, closed ended questions using a

five point scale (strongly disagree, disagree, neutral, agree

and strongly agree) were used to collect specific feedback on

three main aspects, including features, benefits, and tech-

nical performance of the electronic portfolio system. The

items in the first two groupings were developed using sec-

ondary data from the literature on (electronic) portfolios

[46,9,12,13,1618]. Those in the last category were adapted

from the user-perceived web quality instrument developed

by Aladwani and Palvia [19]. Open questions were developed

to gather general comments on the electronic portfolio prac-tice and the think-aloud technique was conducted with five

volunteers from the user group as a pilot test. They were

required to verbalize their thoughts and perceptions when

completing the questionnaire, so as to identify any unclear

or ambiguous questions. This further ensured its validity

[20].

The questionnaires collected from students and staff were

analyzed separately. Although the same questionnaire was

used forboth studentsand staff,theywerepart of two cohorts.

The mean was used to determine the central tendency and

variability was measured using standard deviation (SD) of

each closed-ended question. Content analysis was used for

the open questions of the student set with quasi-statistics asan accounting system. The proportions of the frequency were

then putinto tables for data presentation. The responses from

open questions by staff have been used as quotations in the

discussion as necessary [20,21].

5. Status report

5.1. System overview

Using the proposed model in Fig. 1, a tailored web database

portfolio system (http://radep.hti.polyu.edu.hk ) was devel-

oped. There is a common login page to direct users to author(Fig. 2), reviewer or administrator mode basedon their creden-

tials, and to requestpasswords if forgotten. The functionalities

of the author and reviewer modes are similar. However, a

Build Portfolio area is provided in the author mode and it

becomes Grade Portfolio in the reviewer mode. One addi-

tional function, Memo, is also included in the reviewer view.

It is an electronic space for reviewers to put down their own

experiences and feelings from their teaching, for example,

and to accumulate such to provide an evaluation of their

teaching performance over a period of time. The administra-

tor interface is for granting special permission to users and

maintaining contentsof the system such as competence state-

ments archived in the database.
http://radep.hti.polyu.edu.hk/http://radep.hti.polyu.edu.hk/


4/13


Fig. 2 A snapshot of the Build Portfolio function of the author view.

5.2. Build Portfolio

Build Portfolio is one of the most important functions of this

web database portfolio system. This allows users to conve-

niently develop their portfolio. In Fig. 2, the first rectangle

highlights the main menu. Users can switch to other func-

tions by clicking the appropriate links. The second rectangle

shows the sub-menu (menu of the Build Portfolio function)

that allows users to select which stage and competence cat-

egory they would like to work on. The bottom section is the

action menu generated by contents from thedatabase andthe

ASPscript provides a framework for itsrendering.There is one

heading and four columns. The Competence column showsthe corresponding competences required for that stage under

the selected category. Criteria further defines the expected

levels of development. The Portfolio Covering Letter and

Reflect column provide links to reflection input forms and

the Upload Evidence directs users to the file upload page.

When a user wants to build a portfolio, the appropriate

stage and competence categories are selected in the main

menu. Evidence of development against the competence cat-

egory/statements should be provided first through accessing

the links under the Upload Evidence column. A popup win-

dow with probing questions for collection and selection of

evidence is shown together with the file upload page. This

reinforces users to review their experience for the selected

area, to collect evidence that can represent it, and choose

the best evidence out of the collection. Normally, a maximum

of two files can be uploaded for each area, so as to sup-

port the evidence selection process. If the previous uploaded

files occupy the quota, accommodation of new files can be

achieved through replacing the previous ones. Also, permis-

sion for increasing the quota to a maximum of three can be

granted through the administrator interface if the user has a

strong reason.

The file upload page also has a link to the Q/R interface for

DICOM images from PACS. This interface uses a patient root

Q/R information model. Clients can input patient level keys

such as patient name, patient ID, date of birth and study dateto query images [22]. These HTTP inputsare then converted to

the DICOM standard through a third party Component Object

Model (COM) object, i.e. the Q/R broker called by the ASP page.

The Q/R broker becomes the Service Class User (SCU) of the

C-Find service and the Service Class Provider (SCP), i.e. the

PACS server sends back the query result. Again, it is translated

backto HTTP responses. The subsequent retrieval procedure is

translated to C-Move and C-Store services by the Q/R broker.

Eventually, the required DICOM images are sent to the web

server. Another function of the Q/R broker is to convert the

retrieved DICOM files to Joint Photo-graphics Experts Group

(JPEG) format for browser display and archiving in the web

database [15].


5/13


Fig. 3 A snapshot of the retrieved images from PACS in JPEG format for display and further archiving.

Fig. 3 shows a snapshot of the Q/R interface displaying

the retrieved Magnetic Resonance (MR) images of a patient

called PACS Interface Demo taken on 10/07/2004 in JPEG for-

mat. The first rectangle shows the competence area that the

images are for. Following is a hyperlink to the file upload

page. The third rectangle highlights the information about

the images shown underneath it. The images will be archived

as evidence for the competence area once the thumbnail is

clicked. The last rectangle lists the image file archived in this

way.

The last step in the portfolio building is reflective writ-

ing, using the reflection input form. Again, a browser window

with guiding questions for reflection pops up together withthe form when the reflect hyperlink is clicked. It explains to

authors that reflection is the written account/justification of

the evidence provided and it should include responses to the

following questions:

1. How do youthink theexperienceselectedshows your com-

petence development for this area?

2. Do you see any area requiring further development based

on the experience?

3. How can you improve your competences next time?

4. From this evidence what do you see as your particular

strengths?

In Fig. 4, a snapshot of a reflection input form is shown.

The heading is the area for reflection. On the left hand side

is a text area for users reflection input. Opposite is a column

headed Evidence Insertion listing out the evidence archived.

These are buttons to insert hyperlinks of corresponding evi-

dence files into the text area using JavaScript. At the bottom,

there are buttons for saving and submitting the reflection.

When a user finishes building a portfolio, it is necessary to

click the submit button in the left lower corner of the Build

Portfolio page to declare it is ready for review. The electronic

portfolio is then locked. The hyperlinks of the file upload page

and reflection input forms are hidden, and access to them is

denied to block anysubsequent modification. Also, automatede-mails are generated for notifying the responsible reviewer

and informing the user of the submission status.

5.3. Grade Portfolio

The Grade Portfolio is the most important function of the

reviewer interface. As shown in Fig. 5, there are filter and

search utilities for locating authors portfolios. The query

result is a list of hyperlinks to corresponding authors portfo-

lios. The bottom browser is an authors portfolio called Grade

Portfolio Form as it has utilities for providing levels of attain-

ment andfeedback. The layout is similar to the authors Build

Portfolio page. The major difference is an absence of the


6/13


Fig. 4 A snapshot of a reflection input form.

criteria column which is substituted by two new columns,

i.e. grade and comments. Again, the portfolio is divided into

different categories. There is a checkbox called Benchmark

Guide to aid the reviewer in providing levels of development.

After checking this box, the expectedlevels of development, as

shown in the criteria column of the author mode, are filled in

the grade column. This provides a reference for the reviewer

to determine the attainment level and saves reviewing time

as it is only necessary to make the necessary adjustment. The

symbols under the reflection and evidence columns signify

that there is reflection andevidence forthe area as hyperlinks.

The utilities under the grade and comments columns enable

the reviewer to provide the levels of attainment and feed-back after reviewing the materials. Eventually, the checkbox

Finished Review? is checked to send out automated e-mails

notifying authors of the availability of feedback.

5.4. View Portfolio

The View Portfolio function aims to increase the readabil-

ity of portfolios for authors and reviewers. In reviewer mode,

there are filter and search utilities to locate relevant portfolios

(Fig. 6). Filter parameters include year of study, completion,

overall grade and problems with portfolios. This is similar to

the Grade Portfolio interface. The search results, i.e. a list of

authors names, serve as hyperlinks to corresponding port-

folios, i.e. View Portfolio Form. The general layout is also

similar to the GradePortfolio Form. However, grade andcom-

ments cannot be modified in this interface. If the relevant

contentssuch as reflection, evidence, grade andcommentsare

present in the portfolio, corresponding information or hyper-

links are shown. For example, there is an overall comment for

the portfolio shown in Fig. 6 but not elsewhere. There is no

hyperlink under the comments column. Also, the font colour

of the grades change according to their values. Red is used to

highlight an attainment below standard while blue indicates

outstanding achievement. Grey represents appropriate devel-

opment progress. In author mode, the View Portfolio Form is

named View Portfolio and the interface is more or less thesame.

5.5. Other utilities

Other utilities of the system include an introduction, user

guide, internal message, settings changing and logout. In the

introduction, the portfolio pedagogy and its implementation

settingsare presented. The userguide serves as a quick techni-

cal reference. The internal message utility provides a channel

for text communication between authors and reviewers. Its

advantage over e-mail is to centralize the communication, so

as to make it more organized and holistic over time. It seems

that e-mail is not an appropriate choice because messages for


7/13


Fig. 5 A snapshot of the Grade Portfolio function.

this purpose may scatter over thewholemailbox andmix with

other e-mails or even be deleted after a period of time. Auto-

mated e-mail notification will fire once there is any message

for a user. Changes of password and an option of receiving

automated e-mail can be made inside the settings changing

utility and the logout function is to foster the system secu-

rity.

5.6. Findings of the ePortfolio questionnaire

Sixty-twoquestionnaireswere collected from the student par-

ticipants, yielding a response rate of 83.8% (62 out of 74). The

three reviewers returned their responses to their question-naires, yielding a 100% response rate.

Table 1 shows only two questions (Q.13 and Q.24) that

have mean values below three (neutral) while the remaining

questions have positive responses as noted in the students

column. However, when a more stringent approach is used

by rounding up the mean to integers, Q.11, Q.12, Q.13, Q.16,

Q.18 and Q.24 should be considered as neutral responses.

The students have provided a positive response to most of

the issues raised in the questions. The first four questions

relate to the Benefits of the ePortfolio (Q.11, Q.12, Q.13 and

Q.16) including development of personal and professional

attributes and higher order skills, assurance of education

quality, strengthening of teacherstudent relationships and

facilitation of collaborative learning. The other two questions

(Q.18 and Q.24) relate to the Technical Performance of the

ePortfolio including ease of use and attractive layout. Using

the same rounding up approach, there were four questions

(Q.4, Q.13, Q.16 and Q.24) which have a neutral response from

the reviewers perspective while a positive response is noted

for the remaining issues.

6. Lessons learned

6.1. Alignment with the portfolio pedagogy

A good web database portfolio should align with the central

underlining values of portfolio pedagogy, which are collect,

select and reflect. They are the three most important fea-

tures of portfolio pedagogy, as set out below [9].

1. A collection of materials from an individuals experiences.

2. A selection of materials from the collection with annota-

tions to justify the selection action which is the outcome

of reflective thinking.

3. Reflection to narrate an individuals development process

and attainment through the contents of portfolio for eval-

uation.

4. Presumption of development.


8/13


Fig. 6 A snapshot of the View Portfolio function.

5. Diversity of contents along and across portfolios.

6. A means of communication to readers of an individualsprogress and attainment.

7. An evaluation tool ([16], p. 130).

In this system, a popup window with guidelines and prob-

ing questions of collect, select and reflect are shown when

authors access file upload pages and reflection input forms.

Although covered in the reflection training workshop, this

technical arrangement reminds and reinforces these impor-

tant aspects. The limitation in the number of files to be

uploaded is another measure to foster authors to exercise

their reflective thinking when making their selection. The

Evidence Insertion functionality in the reflection input form

linksthe individual uploaded artifacts with the reflective writ-ing, so as to increase the coherence of the portfolio as a

whole. This arrangement removes the major criticism of web

database portfolio systems, i.e. the standardization of the

portfolio building process shifts the authors role to that of

content provider or uploader of artifacts, leading to a loss

of reflection [9]. The authors are required to state how they

think the experience selected demonstrates their competence

development for this area followed by the JavaScript gener-

ated hyperlinks to corresponding evidence, and achieves the

real spirit of the selection process as noted above [9,10,16].

Together with reflective comment to suggest attainment and

further improvement, this forms the presumption of develop-

ment. This realizes the hypertext reflection in a web database

portfolio environment which helps to contextualize and sup-

port issues to be justified, which is difficult to accomplishin a paper-based portfolio [23]. Accommodation of diversity

of contents along and across portfolios is achieved through

the system tolerance to different uploaded file formats from

traditional JPEG, Portable Document File (PDF) to DICOM. Var-

ious reflection input forms such as portfolio covering letter,

reflection form for competence category and for individual

statements, also provide flexibility for different users. The

portfolio covering letter serves to present an organized, over-

all picture of development, which is an efficient channel for

both authors and reviewers. However, detailed reflection is

best noted using individual reflection forms. Combined use of

the three channels accommodates a great variety of contents.

The Grade Portfolio function provides a convenient interfacefor portfolio evaluation by use of filter and search utilities

to locate authors portfolios, automated generation of hyper-

links to highlight submitted contents, and Benchmark Guide

to facilitate the grading process. Communication of attain-

ment to readers is the major use of the View Portfolio page,

enhancing the readability of portfolios through functions such

as filter and search utilities, automated hyperlinks generation,

and colour highlighting of development. Other communica-

tion functionalities include internal messages and automated

e-mail generation after portfolio submission, evaluation and

receipt of messages. However, functionalities other than for

portfolio practice are absent in the system, to eliminate any

branching out effect.


9/13


Table 1 A summary of findings of the closed-ended questions.

Questions Student (n = 62) Reviewer (n = 3)

Meana SD Meana SD

Features of the ePortfolio

1. It serves as a collection of materials (referring to the collect procedure) from

an individuals experiences

3.87 (4) 0.61 4.67 (5) 0.58

2. It involves a selection of materials from the collection (referring to the select

procedure)

3.82 (4) 0.59 3.67 (4) 1.53

3. It contains annotations (written accounts) to justify the selection action,

based on the outcome of the individuals reflective thinking

3.69 (4) 0.53 4.33 (4) 0.58

4. It contains reflections indicating an individuals development process and

attainment, following evaluation of these contents

3.81 (4) 0.70 3.33 (3) 1.15

5. There is a presumption (self-evaluation) of development 3.98 (4) 0.61 4.00 (4) 0.00

6. It is capable of accommodating diversity of contents, such as different kinds

of evidence over time and covering different development issues

3.73 (4) 0.61 4.67 (5) 0.58

7. It provides a means of communication with readers on an individuals

progress and attainment

3.52 (4) 0.67 4.00 (4) 1.00

8. It serves as an evaluation tool 3.85 (4) 0.72 4.33 (4) 0.58

Benefits of the ePortfolio

9. It promotes reflective learning 3.63 (4) 0.79 4.00 (4) 1.00

10. It facilitates recognition of strengths for mapping of career pathways 3.50 (4) 0.72 4.33 (4) 1.1511. It facilitates development of personal and professional attributes and various

higher order skills, such as self-motivation and critical thinking

3.48 (3) 0.83 4.00 (4) 1.00

12. It assures education quality 3.39 (3) 0.86 4.67 (5) 0.58

13. It strengthens teacherstudent relationships 2.92 (3) 0.77 3.33 (3) 0.58

14. It illustrates competence development 3.51 (4) 0.65 4.00 (4) 1.00

15. It supports resume writing and continuing professional development (CPD) 3.65 (4) 0.66 4.00 (4) 1.00

16. It facilitates collaborative learning (involving sharing of portfolios with peers,

and providing comments and feedback on others works)

3.42 (3) 0.74 3.33 (3) 0.58

Technical Performance of the ePortfolio

17. It is a secure platform 3.87 (4) 0.71 4.00 (4) 0.00

18. It is easy to use 3.40 (3) 0.98 4.33 (4) 0.58

19. It is always reliable and available 3.50 (4) 0.94 4.33 (4) 0.58

20. Its speed of page loading is fast 3.82 (4) 0.78 4.00 (4) 1.00

21. It is interactive and generates dynamic contents 3.50 (4) 0.72 3.67 (4) 0.58

22. It is flexible for accommodation of your work 3.52 (4) 0.76 3.67 (4) 0.58

23. Its organization of contents is clear 3.56 (4) 0.80 3.67 (4) 0.58

24. Its layout is attractive 2.84 (3) 0.71 2.67 (3) 1.53

a Legend: 1 = Strongly disagree; 2= Disagree; 3= Neutral; 4 = Agree; 5 = Strongly agree (mean rounding up to integers).

6.2. Discussion of the current electronic portfolio

practice

Positive mean scores were noted with respect to Features

of the ePortfolio in the ePortfolio Questionnaire (Q.s 18,

except Q.4 from the reviewer perspective, Table 1). This not

only indicated good alignment of the system functionalities

with portfolio pedagogy, as suggested in Section 6.1, but alsoensured that quality portfolio processes were in place in the

current electronic portfoliopractice. Thisis because a success-

ful portfolio process should posses the features of portfolio

pedagogy, such as collect, select and reflect of evidence of

development [9,11,24]. The only neutral response (Q.4) from

the staff findings could be explained by one reviewers sug-

gestions for improvement:

There should be on-going (embedded in their learning,

in the form of reflective diary) documentation of their

learning experiences rather than an overall retrospective

summary after the semester. In this way, it may provide

quality data. (Reviewer of First Year Portfolios)

From that reviewers perspective, the reflections contained

in the electronic portfolios cannot indicate an individuals

development process since, it is suggested, this is only an

overall retrospective summary after the completion of the

academic year. The reviewers comments are understandable.

The most important issue is that the accumulated reflections

throughout the three years of study should be able to demon-

strate an individuals development process.It isuseful torequeststudents tonote downtheirimportant

experiences throughout the year as is common practice with

keeping a reflective journal, as they may forget some issues

over a longer period of time [8]. However, the point is whether

it is necessary to expose all experiences without a selection

process. If this is the case, the reviewers may be overloaded

with reviewing quantities of students reflections. It is also not

good for the students development of portfolio building skills

since the ability to select is one of the essential elements of

portfolio building, not only evidence but also reflection. The

reflection which the reviewer works on is actually the final

masterpiece of the students selection effort [4,9]. However, it

maybe useful to providean electronic reflective journal facility


10/13


in the system where students are encouraged to put down

thoughts during the year, and from which they select their

final masterpiece.

Although only four benefits have been perceived by stu-

dent participants, as noted in the Benefits of the ePortfolio

(Table 1), the most important ones promotion of reflective

learning and illustration of competence development were

identified in the responses. The capability of electronic port-folio practice for competence diagnosis is thecore component

of the study aim, while the reflection element is essential for

this practice to succeed [9,13,24,25]. Identification of the two

elements as benefits further confirms the success of the cur-

rent practice. The importance of reflection is also noted in a

reviewers response to the open question, the Most Successful

Aspects of this Exercise/System:

. . . ability to encourage students to be reflective learners.

(Reviewer of Third Year Portfolios)

Students and reviewers responded positively to Technical

Performance of the ePortfolio (Table 1), and positive findings

were noted in the items: security of platform, reliability andavailability, fast page loading, interactivity and generation

of dynamic contents, flexibility for accommodation of work,

clear organization of contents. One more positive response

was shown in the reviewers findings for this category, that

it was easy to use (Q.18, Table 1). Further positive responses

to the open questions from the reviewer of the third year

portfolios are noted below:

Benefits Gained from the ePortfolio Exercise:

. . . it is easy to use. (Reviewer of Third Year Portfolios)

the Most Successful Aspects of this Exercise/System:

. . . ease of use. (Reviewer of Third Year Portfolios)

Suggestions for Improvement:

Use of the portfolio does require a good introduction,

although once this has been done, it is easy to use.

(Reviewer of Third Year Portfolios)

The reviewer of the third year portfolios suggested that

easeof use depended on a goodintroduction. The introduction

for reviewers took place on a face-to-face, individual basis in

contrast to a group briefing session for student participants.

This may account for the students neutral response to the

issue of ease of use (Q.18). Reinforcement of technical issuesmay be necessary for students in the training workshop.

6.3. PACS connectivity

The electronic portfolio system established in this study also

provides PACS connectivity for retrieval of medical images

from any PACS. It seems its importance is not significant for

pre-registration education as students assessment outcomes

can serve as readily available evidence to demonstrate their

professional attainment. However, when extending the use

of the electronic portfolio system from pre-registration edu-

cation to later CPD, it seems the workplace outcomes, for

example, medical images produced in the radiography dis-

cipline, should also be considered as an indicator of their

professional development. The use of this is a derivative of

the competence assessment, medical record review. The med-

ical records, as the readily available evidence of clinical skills,

can be used to infer individuals holistic competence [26,27].

It is noted that the mechanism of the portfolio is to tie up evi-

dence from multi-dimensional assessmentstrategies [13]. The

current system provides a one-stop solution for radiographersto retrieve their evidence of professional development in the

workplace, i.e. medical images produced. These images can be

saved into the system as evidence of development for portfo-

lio building. This is the collection and selection of evidence of

development. Reflective writing can then be provided to jus-

tify their professional development supported by the retrieved

images, thus linking the evidence together. Although this can

be achieved by obtaining images from the web based PACS

and uploading them to the ePortfolio system, the importance

of this integration is to provide automation or ease of use, so

as to promote the use of workplace outcomes as evidence for

portfolio building in theCPD situation. New features have now

been added to the DICOM standard, such as integration withinputs other than traditional radiological images, for exam-

ple, visible light images and waveforms. The use of the PACS

interface is not limited to radiography/radiology but can also

serve other related health care disciplines for which these

objects can provide evidence of workplace outcomes. Using

PACS connectivity may create possible breaches of patient pri-

vacy, so user-authentication of the system and conversion of

DICOM images received to JPEG are the measures in place to

safeguard patient confidentiality and prevent abuse by users.

User-authentication can validate the identity of users and

limitaccessibility to certain images archived in the PACSbased

on their profiles. The entire conversion procedure is carried

out in the system server and this prevents users from obtain-ing the original DICOM files. In effect, users can only receive

those JPEG imagesfrom whichall patient identifiers have been

removed. The conversion of DICOM to JPEG files also facilitates

image distribution through the internet in terms of transmis-

sion speed and compatibility [14,15].

6.4. Study advancement

This tailored web database portfolio system addresses

the major criticism of existing products, which is over-

standardization leading to a shifting role of those building

the portfolio from authors to content providers or uploaders

and hence losing reflection capabilities [9]. A balance betweenstandardization and flexibility is achieved in several ways.

The standardized components include the fixed interfaces for

portfolio building, viewing, grading and internal communica-

tion. These serve as the framework to render contents such

as competence standards and users data into a meaningful

sense. It is crucial to standardize these elements to ensure

efficiency with little interference to authors reflection activ-

ities. Its effect on efficiency over the existing web portfolio

practice, i.e. portfolios housed in a personal homepage style,

includes time saving to learnhomepage creation and determi-

nation of its layout [9,11,28]. Maintenance and administration

of portfolio practice is also another benefit of standardization,

so as to retain its original aim, i.e. to serve a large population


11/13


Fig. 7 A snapshot of the electronic portfolio built through the use of CPD Now.

[9,12]. As the standardized components are those provided

by the stakeholders, such as education institutions and regis-

tering authorities, this approach should not hinder reflection

practices. However, the most important function of the stan-

dardized components is to trigger and facilitate self-directed

reflective thinking. The remaining functionalities should be

flexible enough to facilitate this.

It is suggested that the portfolio is an effective and

efficient way to measure and record individuals personal

achievements and professional accomplishments in terms

of occupational standards and competences. Through crit-

ical reflection on learning experiences, individual, separate

evidence of professional and competence development areintegrated and organized into a meaningful sense, i.e. a sim-

ple, clear and manageable display of attainment which is

useful for stakeholders and students/practitioners [13,17].

This forms the foundation for designing flexible tools to facili-

tate reflection. The Evidence Insertion utility of the reflection

input forms is developed based on this agenda. It enables

authors to use their reflective writing to link up with individ-

ual artifacts into a holistic masterpiece through cyberspace.

This technical arrangement inside a web database portfo-

lio system is an innovative way to address the criticisms

of web database portfolios including role shifting, hypertext

reflection of web portfolios, and the contribution of portfo-

lio pedagogy in the competence movement [9,13,17,23]. Other

flexible tools include capability to accommodate a variety of

uploaded file formats and choices of various reflection input

forms for different purposes. Together with the innovative

development of PACS connectivity in a web database portfo-

lio system, an advanced portfolio practice for undergraduate

medical and health education, and CPD, should be achieved.

For reinforcement of the claim of innovation, the CPD

Now system developed by the influential body of the radio-

graphy profession, the Society and College of Radiographers,

UK, can be used to illustrate this issue. To use the UK system

for portfolio building, the user needs to go to My CPD (port-

folio building) area and create his/her own CPD Framework.

Under thisframework, the usersubmitsthe learning plans andevidence by putting down relevant information for the differ-

ent fields inside the standard templates. The major technical

difference between the templates for learning plan and evi-

dence, is the file upload functionalityprovided in the evidence

template. After completion of this part, the portfolio building

process is completed and the user can go to the My Portfo-

lio (portfolio viewing) area to view his/her portfolio, as noted

in Fig. 7. The electronic portfolio contains three main parts,

My Personal Details, My Progress and Download My Careers.

Under theMy Progresssection, there are three items including

My CPD Evidence, My Learning Plans and My Certificate. The

My CPD Evidence and My Learning Plans are the core contents

of the portfolio which are lists of inputs from the templates


12/13


of learning plans and evidence. In Fig. 7, My CPD Evidence

is shown. It lists the previous evidence input in a sequential

order and without any access to the supporting evidence files

(attachments) [29,30]. This is a typical example of the existing

webdatabase portfoliosystems. The portfoliobuildingprocess

is greatly restricted by the template, i.e. over-standardization

leading to role shifting [9,12]. Also, hypertext reflection can-

not be achieved [23]. Furthermore, PACS connectivity is notavailable inside this system [29,30].

7. Conclusion

Evidence to date suggests the system supports users to build

their portfolios with sound hypertext reflection under a facil-

itators guidance, and with reviewers to monitor students

(authors) progress and provide feedback and comments

online in a programme-wide situation. Positive mean scores

were noted with respect to Features of the ePortfolio in

the ePortfolio questionnaire. These responses indicated good

alignment of system functionalities with portfolio pedagogy

and quality portfolio processes. The web database portfolio

systems format and use seem to have been well accepted by

students and reviewers.

On face value it may appear that the use of the electronic

portfolio system is not closely related to retrieval of clini-

cal data from PACS, but the clinical data is an integral part

of any practitioners workplace outcomes and development.

Ways must be provided to enablesuchevidence to form part of

their portfolios, for example, by using the file upload facility.

The provision of PACS connectivity in the current electronic

portfolio system has illustrated the possibility of providing a

more convenient way to address this issue. Subsequently, this

becomes an integral part of the development of an advanced

electronic portfolio system for practitioners, and goes further

than merely addressing the major criticisms of existing prod-

ucts.

8. Future plans

The outcomes of the electronic portfolio system with PACS

connectivity once fully established could be extended to mon-

itor CPD of clinical radiographers and other professional

areas, so as to provide evidence of practitioners competences

throughout their professional lives. Although this system is

established for the radiography discipline, the standardized

framework provides ease of substituting relevant competence

standards of other disciplines into the database through the

administrator interface. In thisway, a largepopulation of med-

ical and health care students andworkers can be served, so as

to achieve theultimategoal of web databaseportfolio technol-

ogy without sacrificing the most important value of portfolios,

i.e. reflection. Finally, it is the functionalities of reflectionfacil-

itation rather than powerful or attractive technology that are

the most crucial factors.

r e f e r e n c e s

[1] P. Bridge, D. Eddy, The virtual portfolio: from conception to

reality, J. Radiother. Pract. 5 (2006) 17.

[2] M. Lawson, D. Nestel, B. Jolly, An e-portfolio in healthprofessional education, Med. Educ. 38 (2004) 569570.

[3] G. Duque, A. Finkelstein, L. Winer, S. Gold, Immediacy andefficacy of giving feedback and evaluating medical studentsduring their rotations in geriatric medicine: the McGillelectronic portfolio, J. Am. Geriatr. Soc. 51 (2003)S197.

[4] J. Gordon, Assessing students personal and professionaldevelopment using portfolios and interviews, Med. Educ. 37(2003) 335340.

[5] A. Tiwari, C. Tang, From process to outcome: the effect ofportfolio assessment on student learning, Nurs. Educ. Today23 (2003) 269277.

[6] J.G. Alexander, S.W. Craft, M.S. Baldwin, G.W. Beers, G.S.McDaniel, The nursing portfolio: a reflection of aprofessional, J. Contin. Educ. Nurs. 33 (2002) 5559.

[7] Open University School of Health & Social Welfare, Buildingportfolios, Nurs. Manag. 9 (2002) 3033.

[8] V. Dagley, B. Berrington, Learning from an evaluation of anelectronic portfolio to support general practitionerspersonal development planning, appraisal and revalidation,Educ. Primary Care 16 (2005) 567574.

[9] M. Kimball, Database e-portfolio systems: a criticalappraisal, Comput. Composit. 22 (2005) 434458.

[10] S.L. Ramey, M.L. Hay, Using electronic portfolios to measurestudent achievement and assess curricular integrity, Nurs.Educ. 28 (2003) 3136.

[11] G. Pullman, Electronic portfolios revisited: the efoliosproject, Comput. Composit. 19 (2002) 151169.

[12] T. Batson, The electronic portfolio boom: whats it all about?Campus Technol. (2002) (http://www.campus-technology.com/article.asp?id=6984 Accessed 29 November 2005).

[13] M. McMullan, R. Endacott, M.A. Gray, M. Jasper, C.M.L. Miller,J. Scholes, C. Webb, Portfolios and assessment ofcompetence: a review of the literature, J. Adv. Nurs. 41 (2003)283294.

[14] C.K.C. Ng, P. White, J.C. McKay, Establishing a method tosupport academic and professional competence throughoutan undergraduate radiography programme, Radiography(2007) (available online at http://www.sciencedirect.com).

[15] H.K. Huang, PACS and Imaging Informatics: Basic Principlesand Applications, John Wiley & Sons, Inc., New Jersey, 2004.

[16] K.B. Yancey, Portfolio, electronic, and the links between,Comput. Composit. 13 (1996) 129133.

[17] M. Ryan, K.H. Carlton, Portfolio applications in a school ofnursing, Nurs. Educ. 22 (1997) 3539.

[18] D.O. Weddle, S.P. Himburg, N. Collins, R. Lewis, Theprofessional development portfolio process: setting goals forcredentialing, J. Am. Diet. Assoc. 102 (2002)14391444.

[19] A.M. Aladwani, P.C. Palvia, Developing and validating aninstrument for measuring user-perceived web quality,Inform. Manag. 39 (2002) 467476.

[20] B. Johnson, L. Christensen, Educational Research:Quantitative and Qualitative Approaches, Allyn and Bacon,Boston, 2000.

[21] D.F. Polit, C.T. Beck, Nursing Research: Principles andMethods, seventh ed., Lippincott Williams & Wilkins,Philadelphia, 2004.

[22] M.V. Herk, L. Zijp, Conquest DICOM Server 1.4.11Documentation/MicroPACS Conformance Statement, TheNetherlands Cancer Institute, Amsterdam, 2005.

[23] S. Watkins, World wide web authoring in theportfolio-assessed, (inter)networked composition course,Comput. Composit. 13 (1996) 219230.

[24] R. Mason, C. Pegler, M. Weller, E-portfolios: an assessmenttool for online courses, Br. J. Educ. Technol. 35 (2004) 717727.
http://www.campus-technology.com/article.asp%3Fid=6984http://www.sciencedirect.com/http://www.sciencedirect.com/http://www.campus-technology.com/article.asp%3Fid=6984


13/13


[25] K.A. Karlowicz, The value of student portfolios to evaluateundergraduate nursing programs, Nurs. Educ. 25 (2000)8287.

[26] J.A. Sliwa, K.J. Kowalske, Assessing resident clinicalcompetence, Am. J. Phys. Med. Rehabil. 79 (2000) 468473.

[27] E.S. Holmboe, R.E. Hawkins, Methods for evaluating theclinical competence of residents in internal medicine: areview, Ann. Intern. Med. 129 (1998) 4248.

[28] P.E. McShane, Electronic portfolio: a paperless portfolio forinterns to track accomplishments and market themselves, J.Am. Diet. Assoc. 105 (2005) 37.

[29] Axia Interactive Media Limited, CPD Now: SOR ProfessionalDevelopment, Axia Interactive Media Limited, WestYorkshire, 2007 (http://sor.cpdnow.net/Logon.aspxAccessed9 May 2007).

[30] S. Kelly, CPD Now, Synergy (March 2005) 2123.
http://sor.cpdnow.net/Logon.aspxhttp://sor.cpdnow.net/Logon.aspx

cpd framework

Documents