this is an electronic reprint of the original article. this reprint may … · 2018. 10. 2. ·...

This is an electronic reprint of the original article This reprint may differ from the original in pagination and typographic detail

Please cite the original version

Kirsi Talman Maija Hupli Pauli Puukka Helena Leino-Kilpi Elina Haavisto (2018) The predictive

value of two on-site selection methods of undergraduate nursing students A cohort study Journal of

Nursing Education and Practice 8 (7) pp 12-21

httpsdoiorg105430jnepv8n7p12

CORE Metadata citation and similar papers at coreacuk

Provided by Theseus

httpjnepsciedupresscom Journal of Nursing Education and Practice 2018 Vol 8 No 7

ORIGINAL RESEARCH

The predictive value of two on-site selection methods ofundergraduate nursing students A cohort study

Kirsi Talmanlowast12 Maija Hupli1 Pauli Puukka3 Helena Leino-Kilpi14 Elina Haavisto12

1Department of Nursing Science University of Turku Turku Finland2Clinical nursing and Emergency Care Metropolia University of Applied Sciences Helsinki Finland3National Institute for Health and Welfare Turku Finland4Turku University Hospital Turku Finland

Received November 11 2017 Accepted January 11 2018 Online Published January 29 2018DOI 105430jnepv8n7p12 URL httpsdoiorg105430jnepv8n7p12

ABSTRACT

Nursing programs aim to select students who will succeed in theoretical studies and in clinical practice and who are suitable forthe profession Recent literature has suggested an assessment of cognitive and non-cognitive skills in nursing student selectionThe aim of this study is to compare the predictive value of two on-site selection methods used in nursing student selection namelypsychological aptitude tests and literature-based exams A cohort study was conducted Students admitted to four undergraduateBachelor of Science nursing programs at one Finnish nursing school between 2002 and 2004 (N = 626) were allocated intotwo cohorts based on the on-site selection method Follow-up data was collected at two measurement points (May 2004ndashMay2009) The multimethod data collection included the use of admission archives (entrance exam scores) study records (studysuccess) and a structured self-report questionnaire (knowledge and skills) Statistical data analysis was undertaken According tothe results the two on-site selection methods produced very similar results regarding their predictive value Both of the on-siteselection methods predicted knowledge and skills and study success of nursing students to some extent but only explained asmall proportion of variance To conclude neither of the two on-site selection methods should be used alone when predictingknowledge and skills or study success of nursing students Further longitudinal research is needed to investigate the predictivevalue of various on-site selection methods

Key Words Students Nursing Student selection Student admission Person specification Psychological aptitude testLiterature-based exam Cohort study

1 INTRODUCTION

11 The significance and current state of nursing studentselections

Nursing student selection processes have a huge impact onnumerous higher education institutions and applicants seek-ing a study place in nursing education each year It is re-ported that globally there are approximately 35 millionnurses and midwives working within healthcare[1 2] giving

an estimation of the scale of the student selection processesundertaken within nursing education Nursing student se-lection processes have recently come under scrutiny dueto international concerns of turnover rates in the field[3 4]

increased student attrition rates and concerns over a lackof compassion in the workplace[3 5] It is recognised thaton-site student selection methods provide opportunities toevaluate person-centred skills[3] in a way that ldquoputs people

lowastCorrespondence Kirsi Talman Email kirsitalmanmetropoliafi Address Clinical Nursing and Emergency Care Metropolia University ofApplied Sciences PB 4000 00079 Metropolia Finland

12 ISSN 1925-4040 E-ISSN 1925-4059


firstrdquo[1] Furthermore on-site selection methods most of-ten include the assessment of learningcognitive skills andnon-cognitive skills[6 7] Assessment of learningcognitiveskills include for example testing of literacy numeracy andproblem-solving skills[8ndash12] Recent literature also demon-strates the desirability to assess non-cognitive skills (alsoreferred to as person specification non-cognitive qualities orpersonal attributes) in the nursing student selection[13 14] Itis recognised that the demands of the nursing profession suchas personality[3] professional conduct[15] patient safety andquality of care[3 6] should also guide the student selectionHowever there is lack of evidence supporting the currentlyused on-site selection methods in nursing education[12 16]

Further research is needed with cohorts to confirm the useof person specification[13] and its predictability of study suc-cess

In Finland nursing education has a long history of usingperson specification in student selection processes whichwere dominated by psychological aptitude tests executedby psychologists from the 1940s to the year 2000[16] How-ever since the beginning of the new millennium psycholog-ical aptitude tests have been considered financially burden-ing[17] and have been criticised of mainly de-selecting can-didates[18] Furthermore there has been very little evidenceto support the predictive value of the psychological aptitudetests in the student selection of healthcare-related profes-sions[18] Therefore we wanted to evaluate the predictivevalue of the psychological aptitude tests doing so in compar-ison to another on-site selection method Literature-basedexamination was chosen to represent one of the simplest andmost cost-effective on-site selection methods We recognisedthat investigating the predictive value of the selection meth-ods could have an important economic impact within thenursing field Even though there is very little data gatheredin Finland regarding the student selection costs low predic-tive value or lack of significance are strong indicators of amethod that is not cost-effective and these indicators shouldtherefore be of interest to the government and nursing schooladministration There was also a need for evidence-baseddevelopment of nursing student selections due to social andpolitical reasons Thus the aim of this study was to comparethe predictive value of two on-site selection methods usedin nursing student selection namely psychological aptitudetest and literature-based exam

12 Relevant literature on the on-site nursing student se-lection methods

The on-site nursing student selection methods reportedwithin the last 20 years include the use of selection inter-views[12 19] Multiple Mini Interviews (MMIs)[7 20] admis-

sion essays[21] nationwide entry exams[9 10] standardisedtests[8 11 22] and emotional intelligence tests[3 23 24]

Individual and group interviews are widely used for exam-ple in the United Kingdom because of the requirement fromthe Nursing and the Midwifery Council for face-to-face con-tact with nursing applicants[12 15] In a study by Ehrenfeldand Tabak[19] individual interviews were more often asso-ciated with reduced attrition compared to group interviewspast performance and psychometric testing However theattrition rate remained high (30) even with the use of indi-vidual interviews Furthermore interviewing methods werereported to have somewhat low predictive value for attritionand concerns were raised regarding the objectivity of theinterviewing as a method[19] therefore undermining to sup-port for using interviews as a selection tool[16] In a studyby MacDuff et al[12] lecturers clinical staff and studentsgave their perspectives on interviewing processes They con-cluded that students had very mixed opinions on interviewprocesses and staff reported mixed opinions of the validityand reliability of interview processes Staff aimed to achieveholistic assessments in interview processes taking into ac-count a wide range of important candidate attributes suchas interpersonal skills team working confidence problemsolving aptitude for caring motivations and commitment[12]

MMIs have been used as the on-site selection method inwhich both cognitive and non-cognitive attributes are mostoften assessed in nursing student selection[7 20] The assess-ment of non-cognitive attributes has included attributes suchas empathy ethical insights initiative teamwork[7] integrityinterpersonal skills and compassion[20] In a study by Galeet al[7] MMIs in conjunction with numeracy and literacyscores seemed to significantly predict academic success (as-sessment marks for modules in years 1 2 and 3) Literacymarks only weakly predicted academic success[7]

Admission essays have been used to support admission deci-sions in nursing education Sadler et al[21] used admissionessays to find out what the candidates knew of the nursingprofession and why they were interested in nursing Thegroup essay scores were statistically different between theprogram completers and non-completers The program com-pletersrsquo essays also portrayed internalisation of the role of anurse whereas non-completers described nursing as beingexternal to themselves[21]

A nationwide admission test is used to select nursing studentsamong others in the higher education admission process inItaly The test includes assessment of cognitive skills includ-ing logic and general education mathematics and physicsbiology and chemistry[9 10] In a study by Dante et al[9] na-tional admission test scores were associated with academic

Published by Sciedu Press 13


success The chances of students with good entry examscores to graduate within three years were four times as highas those with lower scores However in a study by Lan-cia et al[10] admission test scores did not show a positivecorrelation with final degree grades or GPA of exam scores

Standardised tests such as the American College Test (ACT)the Health Education Systems Inc exam (HESI) the Na-tional League for Nursing (NLN) Pre-Admission Exam andthe Test of Essential Academic Skills (TEAS) have beenfound to be predictive of academic performance in nursingstudies The ACT NLN and SAT have shown some evidenceof being able to predict the pass-rate of the NCLEX-RN[8]

The TEAS has demonstrated that it is able to predict firstsemester success among nursing studies[22]

Emotional intelligence tests have been used to assess person-centred skills or personal qualities such as self-awarenessand social interaction in the on-site selection of nursingstudents[3 24] Rankin[3] used the Assessing Emotions Scale(AES) to assess the emotional intelligence of nursing stu-dent applicants (n = 307) at one UK University A signif-icant predictive relationship was found between emotionalintelligence and three program outcomes practice perfor-mance academic performance and retention even after con-trolling for prior academic achievement age and genderPrior academic attainment and overall emotional intelligencedid not correlate indicating that the two predict differentoutcomes[3] Stenhouse et al[24] used the Trait Emotional In-telligence Questionnaire-Short Form and the Schuttersquos Emo-tional Intelligence Scale upon entry to adult mental healthand midwifery programs (n = 598) They found that emo-tional intelligence scores were not associated with perfor-mance but social connection scores for students withdrawingand remaining were statistically significantly different

In conclusion there is a scarce amount of evidence of thepredictive value of nursing student selection methods specif-ically to assess non-cognitive skills of the applicants

2 METHODS21 AimThe aim of this study is to compare the predictive valueof two on-site selection methods used in nursing studentselection namely psychological aptitude test and literature-based exam The ultimate goal of the study is to provideevidence-based suggestions for nurse educators on futureon-site selection methods Three research questions wereposed

(1) What is the difference in the levels of knowledge andskills of the two cohorts of nursing students at twomeasurement points

(2) What is the difference in the study success of nursingstudents between the two cohorts at two measurementpoints

(3) What factors predict the differences in the levels ofknowledge and skills and study success of nursingstudents between the two cohorts at two measurementpoints

22 DesignThe study design was a cohort study that utilised admissionarchives study records and a self-report questionnaire

23 ParticipantsAll students selected for nursing programs (N = 626 namelynursing midwifery public health nursing) at a university ofapplied sciences (UAS) in Southern Finland between autumn2002 and autumn 2004 were considered eligible to partic-ipate in the study Students selected for admission to theprograms based on a psychological aptitude test (includingteacher interviews) (3 intakes autumn 2002 spring 2003autumn 2003) were allocated to Cohort I Students selectedusing a literature-based exam (2 intakes spring 2004 autumn2004) were allocated to Cohort II

At the time of the study approximately 120 nursing 70 mid-wifery and 40 public health nursing students were acceptedinto the programs each year at the UAS in question A psy-chological aptitude test with teacher interviews had been thedominating on-site selection method and was carried out forthe last time in autumn 2003 The UAS changed the on-siteselection method to a literature-based exam in 2004 to evalu-ate theoretical abilities and motivation of the applicants morecost-effectively As part of the revision the psychologicalaptitude test was omitted

24 The psychological aptitude test and literature-basedexam

The psychological aptitude test undertaken by Cohort I (years2002 2003) comprised six parts learning abilities studystrategies personality testing (Wartegg I amp II Sentence Com-pletion Test SELF-test) group situation one-on-one inter-view with a psychologist and one-on-one interview with anursing teacher The maximum score given was 55 points Inpreparation for the literature-based entrance exam applicantsneeded to read a nursing ethics book which included themessuch as fundamentals of nursing ethics theory of ethics eth-ical decision-making ethical challenges in nursing ethicalprinciples regarding patients and nursing ethics in variousclinical contexts ethical guidelines and ethics from a soci-etal perspective The literature-based exam undertaken byCohort II was developed and piloted in 2003 The final ver-sion of the exam comprised 130 statements and applicants

14 ISSN 1925-4040 E-ISSN 1925-4059


indicated whether the statements were correct or incorrect orif they did not know the answer Applicants were awarded 05points for each correct answer -05 points for each incorrectanswer and ldquodonrsquot knowrdquo responses yielded 0 points Thestatements handled topics such as ethical decision-makingethics in general and ethics in research The maximum scorewas 65 points

25 ProcedureNursing students were asked to complete the self-report ques-tionnaire during their tutorial classes in their third semester(Time 1) and during their final semester (Time 2) The thirdsemester was chosen as the first measurement point becausethe majority of the attrition in the nursing programs takesplace during the first year of nursing education The longitu-dinal data was collected between May 2004 and May 2009

(see Figure 1)

26 Data collectionThe multimethod data collection process included the useof admission archives study records and a structured self-report questionnaire It was also considered important tocontrol covariates (eg age previous work experience andnursing program) The entrance examination overall examscores were collected from the admission archives of theUAS Study success was measured using study records toestablish grade point averages (GPAs) and study progressGPAs were calculated from pre-determined courses (see Ta-ble 1) Study progress was established using the data on thenumber of students that had stayed in the program (reten-tion) and the number of students that had resigned from theprogram (attrition)

Figure 1 Response rates exclusions and measurement points (Cohort I = students selected with psychological test CohortII = students selected with literature-based exam)



Table 1 Data from the study records

Measurement I Measurement II

Study progress

Clinical practice Number of completed clinical practice credit points

Overall study progress Number of credit points Number of credit points

GPA of pre-determined courses

Nursing-related subjects 1 Basics of nursing 2 Rehabilitative and supportive nursing

1 Basics of nursing 2 Rehabilitative and supportive nursing 3 Acutely ill nursing 4 Mental health and substance abuse nursing 5 Nursing of children and adolescents 6 Nursing of the elderly 7 Adult health promotion and nursing

Research and science- related subjects

1 Introduction to philosophy and research 2 Fundamentals of research I 3 Fundamentals of nursing science

1 Introduction to philosophy and research2 Fundamentals of research I 3 Fundamentals of nursing science 4 Final thesis and publication

Note Measurement I = third semester Measurement II = graduation

The levels of knowledge and skills was measured using theStudent Nursesrsquo Knowledge and Skills (SNKS) questionnairedeveloped by Eriksson Hupli amp Leino-Kilpi for this studyWhile the SNKS questionnaire was being developed a pre-viously developed questionnaire by Solante[25] was utilisedThe instrument by Solante was modified and further devel-oped with the written permission of the author Only one ofthe subscales (interaction with patients) was used as per theoriginal instrument All other subscales were modified andtwo new subscales were added The SNKS questionnairecomprises 138 items and seven subscales theoretical knowl-edge (medical and nursing related) (53 items) patient edu-cation (9 items) professional interaction (16 items) clinicalskills (35 items) decision-making skills (9 items) teamworkskills (8 items) and research and development skills (8 items)Each item was rated with a four-point Likert scale withanswers ranging from strongly disagree to strongly agreeHigher scores indicated higher levels of knowledge and skillsof nursing students The questionnaire took approximately20 minutes to complete The SNKS questionnaire was testedand found reliable using Cronbachrsquos alfa at the first andsecond measurement points the reliability ranging in theseven main subscales from 082 to 093 and 085 to 094consecutively

27 Sample

All in all 413 undergraduate nursing students participatedin the study (see Figure 1) Prospective data were collectedduring the third semester and at the time of graduation Theresponse rates were considered to be high at both measure-ment points (see Figure 1) Out of the 413 participants 201participated in both measurement points (follow-up rate of49) Retrospective data included entrance examinationresults for Cohort I and Cohort II students The entranceexamination results for 14 students were not available Stu-

dents who had completed the pre-determined courses (seeTable 1) and had enrolled for their third or their last semesterwere included in the study Students who had dropped out ofthe program (resigned) were absent during their third or lastsemester had transferred from a different UAS or had notcompleted the pre-determined courses were excluded fromthe study (see Figure 1)

28 Ethical considerationsResponsible ethical conduct was followed throughout eachphase of this study[26] Approval for this research was grantedby the institutionrsquos research committee The data were ob-tained through self-reported data and the institutionrsquos entryexam archives and study records During face-to-face recruit-ment students were informed of the study and were assuredthat participation was voluntary and would not affect theirgrades or progress in the program The recruitment and datacollection took place during tutorial lectures by a member ofthe research team Thus the data collection did not involveteachers who were responsible for teaching the groups Thecohort design required the identification of the students Thenames and university identification numbers were coded tomaintain anonymity of the nursing students The originalquestionnaires were destroyed after the data analysis andreporting of the results

29 Data analysisData were analysed using SPSS software version 180 andSAS version 91 Descriptive statistics were used to describethe sample characteristics Seven categories were formedaccording to the subscales of the SNKS questionnaire tocompare the self-reported levels of knowledge and skillsbetween the two cohorts An overall score of the level ofknowledge and skills was calculated as an average of themean values of the seven subscales Study success was es-tablished through the study progress (clinical practice credit

16 ISSN 1925-4040 E-ISSN 1925-4059


points overall credit points) and the GPAs of pre-determinedcourses (nursing-related research and science-related stud-ies) (see Table 1) Differences in the levels of knowledge andskills and study success between the two cohorts were ex-amined using an unpaired T-test and a Wilcoxon test (whenappropriate) The relationships of sample characteristics toknowledge and skills and study success were examined withan unpaired T-test and a one-way ANOVA with post hoctests (Tukey) to avoid type I errors[27] The relationship ofthe self-reported data and study success was examined withPearsonrsquos correlation coefficient Linear regression was usedto control the effect of covariates regarding the significantdifferences within the two cohorts The selection methodand the sample characteristics that were significantly related

to the level of knowledge and skills and study success wereselected for the linear regression analysis The analysis wasperformed stepwise and continued until all of the includedvariables increased the model prediction significantly Thestatistical test was considered to be significant if the p-valuewas lt 05[27]

3 RESULT31 ParticipantsThe sample characteristics of the nursing students of the twocohorts were examined (see Table 2) At the first measure-ment point (third semester) there were significant differencesbetween the cohorts in age marital status and previous workexperience

Table 2 Sample characteristics of the cohort study

Characteristics

Measurement I

p-value

Measurement II

p-value Cohort I (n = 234) Cohort II (n = 126) Cohort I (n = 149) Cohort II (n = 108)

n n n n

Gender

NS

NS Female 218 932 123 976 145 973 104 963

Male 16 68 3 24 4 27 4 37

Marital status

0179

NS Not married 113 498 45 366 89 601 67 644

Marriedco-habiting 114 502 78 634 59 399 37 356

Number of under aged children

NS

NS Yes 29 124 13 103 21 141 14 130

No 205 876 113 897 128 859 94 870

Nursing program

NS

NS

Nursing 125 534 57 456 70 469 58 552

Public health nursing 38 162 25 200 31 208 24 229

Midwifery 71 303 43 344 48 322 23 219

Other reason 6 26 2 16 5 34 3 28

The level of basic education

NS

NS Elementary school 37 16 11 9 23 15 10 9

High school 8 3 5 4 3 2 4 4

Matriculation exam 188 81 110 87 123 83 94 87

Previous professional education

NS

NS Yes 84 36 35 28 56 38 30 28

No 146 64 91 72 93 62 76 72

Working while studying

NS

NS Yes 88 38 56 45 96 64 72 69

No 145 62 69 55 53 36 33 31

Working outside of nursing while studying

NS

NS Yes 60 26 28 22 22 15 14 13

No 172 74 97 78 127 85 91 87

Previous work experience in nursing

0038

NS Yes 90 39 30 24 54 36 27 25

No 141 61 96 76 95 64 79 75

Consideration of dropping out

NS

NS Oftenquite often 13 6 8 6 9 6 9 9

Sometimes 87 38 42 34 63 42 45 44

Never 127 56 75 60 77 52 48 47 Chi square test NS = p gt05 Measurement I = third semester Measurement II = graduation Cohort I = students selected using a psychological test Cohort II = students selected using a literature-based exam



As shown in Table 2 significant differences in sample charac-teristics were not detected between the cohorts at the secondmeasurement point (graduation) (see Table 2)

32 Differences in the levels of knowledge and skills ofnursing students between the two cohorts

In the third semester (Time 1) significant differences weredetected in two subscales (out of seven) theoretical knowl-edge and clinical skills (Table 3) Students selected usingthe psychological aptitude test (Cohort I) assessed their levelof theoretical knowledge higher (mean 222 SD 029) than

students selected using the literature-based exam (CohortII) assessed theirs (mean 216 SD 032) (p = 04) CohortI students also assessed their level of clinical skills higher(mean 284 SD 038) than Cohort II students did (mean 272SD 041) (p = 01) At the time of graduation (Time 2) dif-ferences between the two cohorts were detected in one of thesubscales (out of seven) that being clinical skills (see Table3) Cohort I students assessed their level of clinical skills(mean 324 SD 038) higher than Cohort II students assessedtheirs (mean 310 SD 044) (p-value = 006)

Table 3 Self-reported levels of knowledge and skills of the two cohorts at the two measurement points

Measurement I

p

Measurement II

p Cohort I (n = 234) Cohort II (n = 118) Cohort I (n = 141) Cohort II (n = 101)

Mean SD Range Mean SD Range Mean SD Range Mean SD Range

Overall score 252 031 164-340 246 033 157-383 NS 297 032 216-373 293 037 159-366 NS

Theoretical knowledge 222 029 160-310 216 032 140-390 04Dagger 273 035 175-368 277 045 142-379 NS

Patient education 227 052 100-356 221 049 133-400 NS 289 047 200-400 281 051 133-389 NS

Professional interaction 278 043 163-381 277 046 144-400 NS 305 039 181-400 303 047 125-394 NS

Clinical skills 284 038 186-377 272 041 160-369 01dagger 324 038 217-400 310 044 183-383 006dagger

Decision-making skills 268 050 100-400 265 045 133-389 NS 320 048 200-400 309 046 200-400 NS

Teamwork skills 298 047 163-400 292 048 138-400 NS 323 046 225-400 316 044 163-400 NS

Research and development skills

228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS

Scale 1-4 (1 = insufficient level of knowledge and skills 4 = sufficient level of knowledge and skills) daggerT-test DaggerWilcoxon test NS = p gt 05 Cohort I = students selected using a psychological test Cohort II = students selected using a literature-based exam Measurement I = third semester Measurement II = graduation

33 Differences in the study success of nursing studentsbetween the two cohorts

During the third semester (Time 1) there were no significantdifferences between the cohorts in study progress (see Table4) However significant differences were detected in theGPAs of the nursing students Cohort II students succeeded

better in nursing-related studies as well as in the researchand science-related studies than Cohort I students At thetime of graduation (Time 2) there were no significant dif-ferences in study progress or in the GPAs between the twocohorts

Table 4 Study success of the two cohorts at the two measurement points


Cohort I (n = 234) Cohort II (n = 126) p



Study progress

Amount of clinical practice credits 24 5 8-41 24 3 9-33 NS 91 20 59-120 87 18 66-120 NS

Overall amount of credits 80 9 39-99 79 9 46-93 NS 236 27 203-272 229 25 190-273 NS

GPA

Nursing related courses 31 06 15-5 33 07 2-5 0033dagger 35 05 23-49 36 05 24-47 NS

Research and science related courses 28 09 1-5 31 09 1-5 0011dagger 32 07 17-50 33 08 18-50 NS Grade Point Averages of theoretical studies (refer to Table 2) Scale 1-5 (1 = satisfactory 2 = very satisfactory 3 = good 4 = very good 5 = excellent) 1 credit = 27 hours of student work NS = p gt 05 daggerT-test Measurement I = third semester Measurement II = graduation Cohort I = students selected using a psychological test Cohort II = students selected using a literature-based exam

34 Predictors of the level of knowledge and skills and

study success of nursing students between the twocohorts

According to the linear regression analysis two factors thestudent selection method and the nursing program most of-ten account for the detected statistical differences between

the cohorts in the levels of knowledge and skills and studysuccess (see Table 5) However the overall variance remainsbelow 11 indicating that neither of the two on-site se-lection methods were particularly predictive of the nursingstudentsrsquo levels of knowledge and skills or study success

18 ISSN 1925-4040 E-ISSN 1925-4059


Table 5 Predictors of the levels of knowledge and skills and GPAs of nursing students using linear regression

Predictor variables

Self-reported level of knowledge and skills GPA of pre-determined courses

Clinical skills Nursing-related subjects Research and science-related subjects

Measurement point Idagger Measurement point IIdagger Measurement point IDagger Measurement point IDagger

p-value p-value p-value p-value

Selection method 0305 0130 0009 0080

Age - - 0012 -

Nursing program - 0396 - 0011

Level of basic education - - - 0006

Previous work experience in nursing 0038 - - -

Model prediction 44 57 54 103

Note Measurement I = third semester Measurement II = graduation Cohort I = students selected using a psychological test Cohort II = students selected using a literature-based exam dagger Cohort I

studentsrsquo level of knowledge and skillsstudy success better than that of Cohort II students Dagger Cohort II studentsrsquo level of knowledge and skillsstudy success better than that of Cohort I students

4 DISCUSSION

41 Main resultsBased on previous literature it has been unclear which on-site selection methods should be used to best predict thestudy success of nursing students[12 13] Therefore this studyaims to compare the predictive value of two on-site selectionmethods on study success of undergraduate nursing studentsFirstly the results of this study demonstrate that there wasvery little difference in the self-reported levels of knowl-edge and skills or the objectively assessed study successof nursing students between the two cohorts This resultis somewhat surprising because we initially assumed thatstudents selected with the psychological aptitude test woulddemonstrate higher levels of knowledge and skills and betterstudy success than students selected with a literature-basedexam This initial assumption was based on the long historyof psychological aptitude tests in Finland and the inclusionof cognitive and non-cognitive elements Because of the lackof previously reported studies on the use of psychologicalaptitude tests in nursing student selection the results of thisstudy cannot be directly compared to previous study results

Secondly the results of this study suggest that both psycho-logical aptitude tests and the literature-based exams can beused to predict nursing performance and study success tosome extent However the results of the linear regressionanalysis demonstrate that the selection method only explainsa small proportion of the variance (44-103) (see Table5) It is recognised that the explanatory power of the twoon-site selection methods might have been higher if all of theapplicants would have been invited to take the entrance examwhich would have created a more heterogeneous group How-ever during the selection process of this study approximatelyone-third of the applicants were invited to take the entranceexam based on their previous academic achievement Thegroup was therefore more homogenous which may havedecreased the explanatory power of the two methods

Thirdly based on the study results it is impossible to con-clude whether future on-site selection methods should con-centrate more on the assessment of cognitive or non-cognitiveskills It would have been interesting to assess the predictivevalidity of the different parts of the psychological aptitudetest such as personality testing (Wartegg I amp II SentenceCompletion Test SELF-test) and learning abilities but un-fortunately we were only able to access the overall scoresof the psychological aptitude test According to the previ-ous related literature a combination of both cognitive[7 9]

and non-cognitive[3 7 24] approaches may be useful in futurenursing student selection However further identification offactors and methods to assess cognitive and non-cognitiveskills of nursing students is required

Fourthly even though the results demonstrate very little dif-ference between the two selection methods the methods varygreatly in cost This cost difference is notable for Finnishnursing programs because the institutions are obligated tocover the entire cost of entrance examinations The appli-cants are entitled to participate in entrance examinations freeof charge[28] To date there is only one publication regardingthe costs of entrance examinations in the social and health-care education in Finland According to Djupsjoumlbacka[17]

in 2002ndash2003 it was estimated that the total cost of a psy-chological aptitude test (including interviews organised byteaching staff) was 8262 euros per applicant whereas thetotal cost of a selection method including a book exam in-terviews organised by teaching staff and a group discussionwas 4440 euros per applicant The estimation of these costsincludes the planning implementation and evaluation of theexamination[17] Currently on-site selection methods varygreatly in Finnish nursing programs According to a sur-vey conducted in January 2014 (answers from 13 out of 23universities of applied sciences) eight of the universitiesof applied sciences were using psychological aptitude testscombined with either an individual interview with a psychol-



ogist or a combination of an individual interview and a groupinterviewsituation One UAS was using a literature-basedexam and four were using literature-based exams combinedwith individualgroup interviews or both At nine of the uni-versities of applied sciences mathematical skills were beingtested alongside other on-site selection methods and at threelanguage skills were being assessed[6] However there is nofurther research data of the predictive value of these selectionmethods

42 LimitationsIn this cohort study the most common sources of bias (in-formation selection and confounding bias) were evaluatedregarding the interpretation of the results Information biasrefers to accurate and consistent measurements and selectionbias to representativeness of the target population[29 30] Inthis study the SNKS questionnaire was found to be a reliableinstrument with an internal consistency ranging from 082to 094 at the two measurement points[31] Selection biaswas reduced by selecting all nursing applicants who wereadmitted to the UAS during the indicated time period of thisstudy Response rates of this study were considered high(see Figure 1) which also reduced selection bias It wascalculated that the study sample represented approximately65 of all Finnish nursing students at the time of the studyThe age and gender of the study sample followed the averageage and gender of Finnish nursing students in general Thusthe study sample can be considered representative of theFinnish nursing student population The study was appliedat one nursing school to minimise the differences in the cur-riculum and study environment but organising the study atone institution may have somewhat limited the study resultsConfounding variables (eg age and nursing program) werecontrolled by using linear regression[30] It was concludedthat the factors selection method and nursing program mostoften accounted for any differences As a limitation to thisstudy 51 of the participants were lost in the follow-up Itis common in prospective cohort studies that all the partici-pants cannot be followed for the entire length of the study[30]

However these losses may affect the validity of the resultsFurthermore the follow-up data was collected between 2004and 2009 which may affect the way the results can be gen-eralised today Even though psychological aptitude testsand literature-based exams are currently used in Finlandthe methods may have developed over the intervening yearsThis needs to be considered when interpreting the resultsHowever in our opinion the time of the data collection doesnot affect the main results of this study Additionally atthe time of the study the nursing curriculum followed the

European core nursing competencies thus increasing thelikelihood of generalisability of the research results at leastin the European context

5 CONCLUSIONSThe findings suggest that the two on-site selection meth-ods are predictive of nursing studentsrsquo levels of knowledgeand skills (psychological test) and study success (literature-based exam) at the beginning of their studies However thefindings also suggest that other factors or on-site selectionmethods should be included to achieve better predictabil-ity It can be concluded that further longitudinal researchis needed to investigate the predictive value of the variouson-site selection methods used in student selection in nursingeducation Future research should include a wider scope ofoutcome measures to better acknowledge the demands of thenursing profession for example by including measures ofsocial emotional and clinical competence Furthermore ob-jective ways to evaluate knowledge and skills or competenceshould be sought after Future research should also focus onthe admissionselection costs to universities We propose thateconomic impact should be investigated more systematicallycollecting at least the minimum set of data presented below

bull Costsndash Planning of entrance exam (eg content of the

exam room reservations photocopying)ndash Implementation of entrance exam (lecturers

other staff)ndash Marking of entrance examndash Feedback for the applicantsndash Expert fees (eg psychologists expertise in a

specific method)ndash Other fees (eg materials venue)

bull Additional information to calculate the costsndash The number of applicants who participated in the

entrance examndash The number of students selected for the programndash Cost per selected studentndash Cost per applicant who participated in the exam

ACKNOWLEDGEMENTSWe would wish to thank Leigh Ann Lindholm for the lan-guage checking of this article

CONFLICTS OF INTEREST DISCLOSUREThe authors declare that there is no conflict of interest

20 ISSN 1925-4040 E-ISSN 1925-4059


REFERENCES[1] World Health Organization WHO Nursing amp midwifery human

resources for health Global standards for the initial education of pro-fessional nurses and midwives 2009 [cited 2015 October 1] GenevaAvailable from httpwwwwhointhrhresourcesstandardsen

[2] World Health Organization WHO Global Health Observatory(GHO) data Density of nursing and midwifery personnel (totalnumber per 1000 population latest available year) 2017 Avail-able from httpwwwwhointghohealth_workforcenursing_midwifery_densityen

[3] Rankin B Emotional intelligence enhancing value-based practiceand compassionate care in nursing Journal of Advanced Nursing2013 69 2717-2725

[4] Flinkman M Salanterauml S Early career experiences and percep-tions ndash a qualitative exploration of the turnover of young regis-tered nurses and intention to leave the nursing profession in Fin-land Journal of Nursing Management 2014 23(8) 1050-7 httpsdoiorg101111jonm12251

[5] Francis R Report of the Mid Staffordshire NHS Foundation TrustPolicy Inquiry The Stationary Office London 2013

[6] Talman K Student Selection in Nursing Education A follow-up studyof two selection methods and their relations to the knowledge skillsand study motivation of nursing students [Doctoral dissertation origi-nal text in Finnish abstract in English] University of Turku AnnalesUniversitatis Turkuensis Series C ndash section 383 2014 Availablefrom httpswwwdoriafibitstreamhandle1002496712AnnalesC383Talmanpdfsequence=2ampisAllowed=y

[7] Gale J Ooms A Grant R et al Student nurse selection and pre-dictability of academic success The Multiple Mini Interview projectNurse Education Today 2016 40(5) 123-127 httpsdoiorg101016jnedt201601031

[8] Stuenkel DL At-risk students Do theory grades + standardizedexaminations = success Nurse Educator 2006 31 207-212httpsdoiorg10109700006223-200609000-00007

[9] Dante A Valoppi G Saiani L et al Factors associated with nurs-ing studentsrsquo academic success or failure A retrospective Italianmulticenter study Nurse Education Today 2011 31(1) 59-64httpsdoiorg101016jnedt201003016

[10] Lancia L Petrucci C Giorgi F et al Academic success or fail-ure in nursing students Results of a retrospective observationalstudy Nurse Education Today 2013 33(12) 1501-1505 httpsdoiorg101016jnedt201305001

[11] Underwood L Williams LL Lee MB et al Predicting baccalau-reate nursing studentsrsquo first-semester outcomes HESI admissionassessment Journal of Professional Nursing 2013 9 S38-S42httpsdoiorg101016jprofnurs201207003

[12] MacDuff C Stephen A Taylor R Decision precision or holisticheuristic Insights on on-site selection of student nurses and mid-wives Nurse Education in Practice 2016 16(1) 40-46 httpsdoiorg101016jnepr201506008

[13] Pitt V Powis D Levett-Jones T et al The influence of personal qual-ities on performance and progression in a pre-registration nursingprogramme Nurse Education Today 2014 34 866-871 httpsdoiorg101016jnedt201408006

[14] Waugh A Smith D Horsburgh D et al Towards a values-basedperson specification for recruitment of compassionate nursing andmidwifery candidates A study of registered and student nursesrsquo andmidwivesrsquo perceptions of prerequisite attributes and key skills NurseEducation Today 2014 34 1190-1195 httpsdoiorg101016jnedt201312009

[15] Nursing and Midwifery Council Standards for pre-registrationnursing education [Internet] 2010 Sept [cited 2017 September 23]

Available from httpswwwnmcorgukglobalassetssitedocumentsstandardsnmc-standards-for-pre-registration-nursing-educationpdf

[16] Taylor R Macduff C Stephen A A national study of selection pro-cesses for students and midwives Nurse Education Today 2014 341155-60 httpdxdoiorg101016jnedt201404024

[17] Djupsjoumlbacka S Selection in an examination National Social andHealth Care Student Selection Project during 2002-2003 [unofficialtranslation original text in Finnish] Diaconia University of Ap-plied Sciences publications 2004 B report 22 Helsinki httpsdoiorg101016jnedt201404024

[18] Rantanen P From selection to workforce Examination of studentselections in the vocational training and in universities of appliedsciences [unofficial translation original text in Finnish] Publicationsby Ministry of Education Ministry of Education Finland Helsinki2004

[19] Ehrenfeld M Tabak N Value of admission interviews in selectingof undergraduate nursing students Journal of Nursing Management2000 8 101-106 PMid11051966 httpsdoiorg101046j1365-2834200000155x

[20] Perkins A Burton L Dray B et al Evaluation of a multiple-mini-interview protocol used as a selection tool for entry to an under-graduate nursing programme Nurse Education Today 2013 33(5)465ndash469 httpsdoiorg101016jnedt201204023

[21] Sadler J Effectiveness of student admission essays in identify-ing attrition Nurse Education Today 2003 23 620-627 httpsdoiorg101016S0260-6917(03)00112-6

[22] Newton SE Smith LH Moore G et al Predicting early academicachievement in a baccalaureate nursing program Journal of Profes-sional Nursing 2007 23(3) 144-149 httpsdoiorg101016jprofnurs200607001

[23] Snowden A Emotional Intelligence and Nurse Recruitment Raschand confirmatory factor analysis of the trait emotional intelligencequestionnaire short form Journal of Advanced Nursing 201571(12) 2936-2949 PMid26265433 httpsdoiorg101111jan12746

[24] Stenhouse R Snowden A Young J et al Do emotional intelligenceand previous caring experience influence student nurse performanceA comparative analysis Nurse Education Today 2015 43 1-9httpsdoiorg101016jnedt201604015

[25] Solante S Healthcare education outcomes Follow-up study for grad-uated midwifes nurses and public health nurses in the year 1990[Masters dissertation unofficial translation original text in Finnish]University of Turku Department of Nursing Science Turku 1997

[26] Responsible conduct of research guidelines [Internet] Finnish Ad-visory Board on Research Integrity 2012 [cited 2017 April 24]Available from httpwwwtenkfienresposible-conduct-research-guidelines

[27] Hair JF Black WC Babin BJ et al Multivariate Data Analysis AGlobal Perspective [7th ed] Boston Pearson 2010

[28] Decree on fees in the University of Applied Sciences [Internet]Finnish Government Decree 2001 2001653 [cited 2016 Jan 16]Available from httpwwwfinlexfifilakialkup200120010653search[type]=pikaampsearch[pika]=6532F2001

[29] Martin MM Epidemiological study designs for health care researchand evaluation in A Bowling S Ebrahim [Eds] Handbook of healthresearch methods Investigation measurement and analysis Maiden-head Open University Press 2005 98-163 p

[30] Healy P Devane D Methodological considerations in cohort studydesigns Nurse Researcher 2011 18 32-36 httpsdoiorg107748nr20110418332c8461

[31] DeVellis RF Scale development Theory and applications [3rd ed]Los Angeles Sage 2012 205 p



ORIGINAL RESEARCH

The predictive value of two on-site selection methods ofundergraduate nursing students A cohort study

Kirsi Talmanlowast12 Maija Hupli1 Pauli Puukka3 Helena Leino-Kilpi14 Elina Haavisto12

1Department of Nursing Science University of Turku Turku Finland2Clinical nursing and Emergency Care Metropolia University of Applied Sciences Helsinki Finland3National Institute for Health and Welfare Turku Finland4Turku University Hospital Turku Finland

Received November 11 2017 Accepted January 11 2018 Online Published January 29 2018DOI 105430jnepv8n7p12 URL httpsdoiorg105430jnepv8n7p12

ABSTRACT

Nursing programs aim to select students who will succeed in theoretical studies and in clinical practice and who are suitable forthe profession Recent literature has suggested an assessment of cognitive and non-cognitive skills in nursing student selectionThe aim of this study is to compare the predictive value of two on-site selection methods used in nursing student selection namelypsychological aptitude tests and literature-based exams A cohort study was conducted Students admitted to four undergraduateBachelor of Science nursing programs at one Finnish nursing school between 2002 and 2004 (N = 626) were allocated intotwo cohorts based on the on-site selection method Follow-up data was collected at two measurement points (May 2004ndashMay2009) The multimethod data collection included the use of admission archives (entrance exam scores) study records (studysuccess) and a structured self-report questionnaire (knowledge and skills) Statistical data analysis was undertaken According tothe results the two on-site selection methods produced very similar results regarding their predictive value Both of the on-siteselection methods predicted knowledge and skills and study success of nursing students to some extent but only explained asmall proportion of variance To conclude neither of the two on-site selection methods should be used alone when predictingknowledge and skills or study success of nursing students Further longitudinal research is needed to investigate the predictivevalue of various on-site selection methods

Key Words Students Nursing Student selection Student admission Person specification Psychological aptitude testLiterature-based exam Cohort study

1 INTRODUCTION

11 The significance and current state of nursing studentselections

Nursing student selection processes have a huge impact onnumerous higher education institutions and applicants seek-ing a study place in nursing education each year It is re-ported that globally there are approximately 35 millionnurses and midwives working within healthcare[1 2] giving

an estimation of the scale of the student selection processesundertaken within nursing education Nursing student se-lection processes have recently come under scrutiny dueto international concerns of turnover rates in the field[3 4]

increased student attrition rates and concerns over a lackof compassion in the workplace[3 5] It is recognised thaton-site student selection methods provide opportunities toevaluate person-centred skills[3] in a way that ldquoputs people

lowastCorrespondence Kirsi Talman Email kirsitalmanmetropoliafi Address Clinical Nursing and Emergency Care Metropolia University ofApplied Sciences PB 4000 00079 Metropolia Finland

12 ISSN 1925-4040 E-ISSN 1925-4059




























14 ISSN 1925-4040 E-ISSN 1925-4059




(see Figure 1)







Study progress











27 Sample





16 ISSN 1925-4040 E-ISSN 1925-4059






Characteristics

Measurement I

p-value

Measurement II


n n n n

Gender

NS

NS Female 218 932 123 976 145 973 104 963

Male 16 68 3 24 4 27 4 37

Marital status

0179

NS Not married 113 498 45 366 89 601 67 644



NS

NS Yes 29 124 13 103 21 141 14 130

No 205 876 113 897 128 859 94 870

Nursing program

NS

NS

Nursing 125 534 57 456 70 469 58 552


Midwifery 71 303 43 344 48 322 23 219

Other reason 6 26 2 16 5 34 3 28


NS


High school 8 3 5 4 3 2 4 4



NS

NS Yes 84 36 35 28 56 38 30 28

No 146 64 91 72 93 62 76 72


NS

NS Yes 88 38 56 45 96 64 72 69

No 145 62 69 55 53 36 33 31


NS

NS Yes 60 26 28 22 22 15 14 13

No 172 74 97 78 127 85 91 87


0038

NS Yes 90 39 30 24 54 36 27 25

No 141 61 96 76 95 64 79 75


NS


Sometimes 87 38 42 34 63 42 45 44









Measurement I

p

Measurement II











228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS










Study progress



GPA







18 ISSN 1925-4040 E-ISSN 1925-4059



Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059































































14 ISSN 1925-4040 E-ISSN 1925-4059




(see Figure 1)







Study progress











27 Sample





16 ISSN 1925-4040 E-ISSN 1925-4059






Characteristics

Measurement I

p-value

Measurement II


n n n n

Gender

NS

NS Female 218 932 123 976 145 973 104 963

Male 16 68 3 24 4 27 4 37

Marital status

0179

NS Not married 113 498 45 366 89 601 67 644



NS

NS Yes 29 124 13 103 21 141 14 130

No 205 876 113 897 128 859 94 870

Nursing program

NS

NS

Nursing 125 534 57 456 70 469 58 552


Midwifery 71 303 43 344 48 322 23 219

Other reason 6 26 2 16 5 34 3 28


NS


High school 8 3 5 4 3 2 4 4



NS

NS Yes 84 36 35 28 56 38 30 28

No 146 64 91 72 93 62 76 72


NS

NS Yes 88 38 56 45 96 64 72 69

No 145 62 69 55 53 36 33 31


NS

NS Yes 60 26 28 22 22 15 14 13

No 172 74 97 78 127 85 91 87


0038

NS Yes 90 39 30 24 54 36 27 25

No 141 61 96 76 95 64 79 75


NS


Sometimes 87 38 42 34 63 42 45 44









Measurement I

p

Measurement II











228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS










Study progress



GPA







18 ISSN 1925-4040 E-ISSN 1925-4059



Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059







































(see Figure 1)







Study progress











27 Sample





16 ISSN 1925-4040 E-ISSN 1925-4059






Characteristics

Measurement I

p-value

Measurement II


n n n n

Gender

NS

NS Female 218 932 123 976 145 973 104 963

Male 16 68 3 24 4 27 4 37

Marital status

0179

NS Not married 113 498 45 366 89 601 67 644



NS

NS Yes 29 124 13 103 21 141 14 130

No 205 876 113 897 128 859 94 870

Nursing program

NS

NS

Nursing 125 534 57 456 70 469 58 552


Midwifery 71 303 43 344 48 322 23 219

Other reason 6 26 2 16 5 34 3 28


NS


High school 8 3 5 4 3 2 4 4



NS

NS Yes 84 36 35 28 56 38 30 28

No 146 64 91 72 93 62 76 72


NS

NS Yes 88 38 56 45 96 64 72 69

No 145 62 69 55 53 36 33 31


NS

NS Yes 60 26 28 22 22 15 14 13

No 172 74 97 78 127 85 91 87


0038

NS Yes 90 39 30 24 54 36 27 25

No 141 61 96 76 95 64 79 75


NS


Sometimes 87 38 42 34 63 42 45 44









Measurement I

p

Measurement II











228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS










Study progress



GPA







18 ISSN 1925-4040 E-ISSN 1925-4059



Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059







































Study progress











27 Sample





16 ISSN 1925-4040 E-ISSN 1925-4059






Characteristics

Measurement I

p-value

Measurement II


n n n n

Gender

NS

NS Female 218 932 123 976 145 973 104 963

Male 16 68 3 24 4 27 4 37

Marital status

0179

NS Not married 113 498 45 366 89 601 67 644



NS

NS Yes 29 124 13 103 21 141 14 130

No 205 876 113 897 128 859 94 870

Nursing program

NS

NS

Nursing 125 534 57 456 70 469 58 552


Midwifery 71 303 43 344 48 322 23 219

Other reason 6 26 2 16 5 34 3 28


NS


High school 8 3 5 4 3 2 4 4



NS

NS Yes 84 36 35 28 56 38 30 28

No 146 64 91 72 93 62 76 72


NS

NS Yes 88 38 56 45 96 64 72 69

No 145 62 69 55 53 36 33 31


NS

NS Yes 60 26 28 22 22 15 14 13

No 172 74 97 78 127 85 91 87


0038

NS Yes 90 39 30 24 54 36 27 25

No 141 61 96 76 95 64 79 75


NS


Sometimes 87 38 42 34 63 42 45 44









Measurement I

p

Measurement II











228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS










Study progress



GPA







18 ISSN 1925-4040 E-ISSN 1925-4059



Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059









































Characteristics

Measurement I

p-value

Measurement II


n n n n

Gender

NS

NS Female 218 932 123 976 145 973 104 963

Male 16 68 3 24 4 27 4 37

Marital status

0179

NS Not married 113 498 45 366 89 601 67 644



NS

NS Yes 29 124 13 103 21 141 14 130

No 205 876 113 897 128 859 94 870

Nursing program

NS

NS

Nursing 125 534 57 456 70 469 58 552


Midwifery 71 303 43 344 48 322 23 219

Other reason 6 26 2 16 5 34 3 28


NS


High school 8 3 5 4 3 2 4 4



NS

NS Yes 84 36 35 28 56 38 30 28

No 146 64 91 72 93 62 76 72


NS

NS Yes 88 38 56 45 96 64 72 69

No 145 62 69 55 53 36 33 31


NS

NS Yes 60 26 28 22 22 15 14 13

No 172 74 97 78 127 85 91 87


0038

NS Yes 90 39 30 24 54 36 27 25

No 141 61 96 76 95 64 79 75


NS


Sometimes 87 38 42 34 63 42 45 44









Measurement I

p

Measurement II











228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS










Study progress



GPA







18 ISSN 1925-4040 E-ISSN 1925-4059



Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059










































Measurement I

p

Measurement II











228 046 113-363 227 045 125-363 NS 284 048 175-400 277 046 175-400 NS










Study progress



GPA







18 ISSN 1925-4040 E-ISSN 1925-4059



Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059






































Predictor variables






Age - - 0012 -







4 DISCUSSION






















20 ISSN 1925-4040 E-ISSN 1925-4059


















































20 ISSN 1925-4040 E-ISSN 1925-4059




































this is an electronic reprint of the original article. this reprint may … · 2018. 10. 2. ·...

Documents