wmftee************************************** · assisted career guidance_evaluation form, pi...

36
..,:t .;:t.,,,,.,..?..... . f.2,.-,:'.,,,....;,..;.... ..,....: ,.:.,..a.7.., , ..., 11,15,,::F.z,... .' ,As'.'.,....,:'_.'zl..frF:.=. ':.:-:.:....'..%1!2'.1.-:?..m._-....k,.,i ..,.e,:4, 62.4.1:44s.'..1;..P.:,. ...,..:I.nt....1.A2..S1::.: ..2.,, , _.?.. DOCUUNT RZSMOI ED 334 484 CG 023 499 AUTHOR Peterson, Gary W.; And Others TITLE A Comparison of the Effectiveness of Three COmpUter-Assisted Career Guidance Systems on College Students' Career Decision Making PrOcesses: Technical Report No. 6. INSTITUTION Florida State Univ., Tallahassee. Center for the SPONS AGENCY PUB DATE NOTE PUB TYPE Study of Technology Development. Kellogg Foundation, Dec 87 40p.; For related reports, see CO 023 498-501. Reports - Research/Technical (143) -- Tests/Evaluation Instruments (160) in Counseling and Career Battle Creek, Mich. EDRS PRICE MF01/PCO2 Plus Postage. DESCRIPTORS *Career Counseling; *Career Guidance; Career Information Systems; College Students; *Computer Assisted Instruction; Counseling Effectiveness; Counseling Techniques; Higher Education IDENTIFIERS Computer Assisted Career Guidance; *DISCOVER System; *Sysam of Interactive Guidance and Information ABSTRACT The Computer-Assisted Career Guidance (CACG) Evaluation Form was developed to evaluate the effectiveness of CACG systems in performing three vital functions in career decision-making. This instrument was subsequently used to compare the effectiveness of DISCOVER, System of Interactive Guidance and Information (SIGI), and SIGI PLUS using 132 subjects from two cohort groups of students in an introductory psychology course. After finishing their assigned system, subjects completed the Computer-Assisted Careor Guidance Evaluation Form, My Vocational Situation, and the Occupational Alternatives Question. Results showed that all three CAW systems were rated positively. However, subjects who expressed a need for career information rated all three CACG systems significantAy more effective in developing and evaluating career options than those subjects who perceived ro need for information. Further, subjects who were undecided about their career direction found SIGI PLUS significantly more helpful for obtaining self-knowledge and occupational knowledge, and more rewarding and enjoyable. The results suggest that perceived effectiveness of CACG systems may be related to the state of client career decidedness and their need for career information. The evaluation instruments and 44 references are included. (Author/LLL) ***************************WMftee************************************** * Reproductions supplied by EDRS are the best that can be made * * from the original document. * ***********************************************************************

Upload: others

Post on 10-Sep-2019

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

..,:t .;:t.,,,,.,..?..... . f.2,.-,:'.,,,....;,..;.... ..,....: ,.:.,..a.7.., , ..., 11,15,,::F.z,... .' ,As'.'.,....,:'_.'zl..frF:.=. ':.:-:.:....'..%1!2'.1.-:?..m._-....k,.,i ..,.e,:4, 62.4.1:44s.'..1;..P.:,. ...,..:I.nt....1.A2..S1::.: ..2.,, ,_.?..

DOCUUNT RZSMOI

ED 334 484 CG 023 499

AUTHOR Peterson, Gary W.; And OthersTITLE A Comparison of the Effectiveness of Three

COmpUter-Assisted Career Guidance Systems on CollegeStudents' Career Decision Making PrOcesses: TechnicalReport No. 6.

INSTITUTION Florida State Univ., Tallahassee. Center for the

SPONS AGENCYPUB DATENOTEPUB TYPE

Study of TechnologyDevelopment.Kellogg Foundation,Dec 8740p.; For related reports, see CO 023 498-501.Reports - Research/Technical (143) --Tests/Evaluation Instruments (160)

in Counseling and Career

Battle Creek, Mich.

EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS *Career Counseling; *Career Guidance; Career

Information Systems; College Students; *ComputerAssisted Instruction; Counseling Effectiveness;Counseling Techniques; Higher Education

IDENTIFIERS Computer Assisted Career Guidance; *DISCOVER System;*Sysam of Interactive Guidance and Information

ABSTRACTThe Computer-Assisted Career Guidance (CACG)

Evaluation Form was developed to evaluate the effectiveness of CACGsystems in performing three vital functions in careerdecision-making. This instrument was subsequently used to compare theeffectiveness of DISCOVER, System of Interactive Guidance andInformation (SIGI), and SIGI PLUS using 132 subjects from two cohortgroups of students in an introductory psychology course. Afterfinishing their assigned system, subjects completed theComputer-Assisted Careor Guidance Evaluation Form, My VocationalSituation, and the Occupational Alternatives Question. Results showedthat all three CAW systems were rated positively. However, subjectswho expressed a need for career information rated all three CACGsystems significantAy more effective in developing and evaluatingcareer options than those subjects who perceived ro need forinformation. Further, subjects who were undecided about their careerdirection found SIGI PLUS significantly more helpful for obtainingself-knowledge and occupational knowledge, and more rewarding andenjoyable. The results suggest that perceived effectiveness of CACGsystems may be related to the state of client career decidedness andtheir need for career information. The evaluation instruments and 44references are included. (Author/LLL)

***************************WMftee*************************************** Reproductions supplied by EDRS are the best that can be made *

* from the original document. *

***********************************************************************

Page 2: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

A Comparison of the Effectiveness of Three Computer-AssistedCareer Guidance Systems on Colltot Students' CareerDecision Making Processes: Technical Report No. 6

by

Gary W. PetersonRebecca E. Ryan-JonesJames P. Sampson, Jr.

Robert C. ReardonMichael Shahnasarian

December, 1987

Project LEARN

Center for the Study of Technology in Counselingand Career Development

Department of Human Services and Studies215 Stone Building

The Florida State UniversityTallahassee, Florida 32306-3001

Gary W. Peterson is an Associat2 Professor, Rebecca Ryan-Jones is a Doctoral Student, James P. Sampson, Jr. is anAssociate Professor in the Department of Human Services andStudies and Robert C. Reardon is Professor and Director of theCurricular-Career Information Service. Michael Shahnasarian is acareer counselor ahd consultant in Tampa, Florida. The third andfourth authors also co-direct che Center for the Study ofTechnology in Counseling and Career Development at Florida StateUniverriity. Support for this investigation was provided by agrant from the W. 7. Kellogg Foundation through Project LEARN.

BEST COPY AVAILABLE 2

U S. DEPARTMENT OF EDUCATIONOn ot E ducal wana$ Researcn and improverneor

EDUCATIONAL C SOURCES INFORMATIONCENTER tERIC)

,Thts dOCurnent tieen relorod.iced as*Clamed from me Perez^ or orpenirerionorvilabNq

r Mr enenoes reeve been made to improvereproduction at:014y

Points of vets or opolionsatatedintnisdocirmitM do not necessarris represent official0E141 POIlhon or poises

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

Crra ry al, RfliersCP

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC).-

Page 3: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

Table of Contents

Pza,Abstract 1

Background 2

Purpose of the Study 2

Development of an Instrument 3

MethodologySubjects 4

Computer-Assisted Career Guidance Systems 4

Procedures 5

Instrumentation 5

Data Coding 7

Data AnalysesResults 8

Discussion 9

References 11Tables 15

Figure 23Appendices

Appendix A Computer Assisted Career GuidanceEvaluation Form 24

Appendix B DISCOVER Progress Record 27Appendix C SIGI Progress Record 28Appendix D SIGI PLUS Progress Record 29Appendix E CACG Questionnaire 30Appendix F Research Participation Release Form 32

a

3

Page 4: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

Table 1

Table 2

Table 3

Table 4

Table 5

List of Tables

ii

Page

Varimax Rotated Factor Matrix for StudentPerceptions of CACG systems 15

Intercorrelations Among Scales of theComputer-Assisted Career Guidance SystemEvaluation Form, Their Means, Stand&rdDeviations and Alpha Reliability Coefficients ... 19

Intercorrelation Matrix 20

MANOVA Summary Table 21

Comparison of the Synthesis Function Accordingto CACG Systems with My Vocational SituationInformation Needs Scale as Moderator Variable ... 22

List of Figures

Figure 1 Comparison of the Syrthesis Function Accordingto CACG Systems with My Vocational SituationInformation Needs Scale as Moderator Variable ... 23

Page 5: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

1

Abstract

The Computer-Assistect Career Guidance_Evalution_Egrm, wasdeveloped to evaluate the effectiveness of CACG systems inperforming three vital functions in career decision-making. Thisinstrument was subsequently used to compare the effectiveness ofDISCOVER, SIGI, and SIGI PLUS using 132 subjects from two cohortgroups of students in an introductory psychology course. Afterfinishing their assigned system, subjects completed the Computer-Assisted Career Guidance_Evaluation Form, Pi Vocational Situation(MVS) by Holland, Daiger, & Power (1960a), and the OccupationalAlternatives Question (OAQ) (Zenor & Schnuelle, 1972). Resultsof the analysis of the data showed that all three CACG systemswere rated positively. However, subjects who expressed a needfor career information rated all three CACG systems significantlymore effective (p < .001) in developing and evaluating careeroptions than those subjects who perceived no need forinformation. Further, subjects who were "undecided" about theircareer direction found SIGI PLUS significantly more helpful forobtaining self knowledge and occupational knowledge(p < .05), and more rewarding and enjoyable (p < .03). Theresults suggested that perceived effectiveness of CACG systemsmay be related to the state of client career decidedness (OAQ)and their need for career information (MVS).

5

Page 6: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

2

Background

User perceptions of the effectiveness of computer-assistedcareer guidance (CACG) systems are among the most common outcomecriteria. While Cairo (1983) and Clyde (1979) have addressed thelimitations of these criteria, Spokane and Oliver (1983) havecriticized the excessive use of self-report measures that lackvalidity and reliability. Nevertheless, user perceptions remainan important outcome variable in CACG research for two reasons;first, such measures may be tailored to match system goals andobjectives; and second, items may be included in the instrumentsto address specific human factors unique to different systems.

As the use of CACG systems has grown, so has the need forinvestigations of the impact of these systems on users In thisregard, Cairo (1983) and Parish, Rosenberg, and Wilkinson (1979)have stressed the need to compare the impact these CACG systemshave on users, particularly in light of the diversity oftheoretical foundations underlying the development of CACGsystems, the ways this technology may be incorporated into localcareer guidance service delivery, and in light of the individualcharacteristics of users.

In addition to ascertaining the impact of the CACG systemsin general, it is also important to realize that the systems maydo different things for different users. Fretz (1981) suggestedthat career decidedness was a potentially important clientattribute for inclusion in vocational intervention studies. Morerecently, Fretz and Leong (1982) hypothesized that careerdecidedness would be "a most logical source of client differencesthat might predict outcomes of career treatment..." (p. 388).Slaney (1980) suggested that the Occupational AlternativesQuestion (OAQ; Zener & Schnuelle, 1972), a measure of expressedvocational interests, could be used as a brief and easilyadministered measure of career indecision and that subjects withdifferent OAQ scores might respond differently to careerinterventions. Slaney (1983) found a clear relationship betweenlevels of career decidedness and responses to careerinterventions for undergraduate females. Another measure ofcareer decidedness, My Vocational Situation (MVS) (Holland,Daiger, & Power, 1980a), has been used by researchers withgenerally positive results (Remer, O'Neill, & Gohs, 1984; Rayman,Bernard, Holland, & Barnett, 1983; Slaney, & Dickson, 1985).

Purpose of the Study

The present study sought to compare the effectiveness ofthree CACG systems, DISCOVER for Schools, SIGI, and SIGI PLUS, interms of (1) their contribution to improved career decision-making and (2) the attitudes of users regarding the helpfulnessof computer-assisted career guidance. The objectives of thisreport are:

Page 7: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

3

(1) to present a valid and reliable self-report measure thatcan be used to compare the relative effectiveness ofCACG systems;

(2) to use the measure to compare the effectiveness of threeselected CACG systems in general; and

(3) to examine whether the effectiveness of CACG Lystems isrelated to entry characteristics of users, namelyvocational identity and career decidedness.

Development of an Evaluation Instrument

An evaluation standard was established to serve as acriterion against which any career guidance system, regardless ofits human and non-human characteristics, could be compared. Thestandard identifies component processes that a guidance systemcould make to enhance career decision-making. The work ofChapman (1975), Gelatt (1962), Harris-Bowlsbey (1983a; 1983b),Katz (1966; 1973), Katz and Shatkin (1983), Sampson, McMahon, andBurkhead (1985), and Super (1973), provided the conceptual basisfor developing the following evaluation standard.

Goal: The goal of any system should be to assist individuals indeveloping career decision-making skills. This may beaccomplished by helping individuals to:

a) develop their career decision-making skills;b) clarify their values, interests, and abilities;c) identify, potentially satisfying occupations congruent

with their values, interests, and abilities;d) acquire an understanding of the world of work;e) integrate their understanding of self, the world of

work, and the needs of significant others so as to makean optimal occupational choice; and to

f) formulate a systematic plan of action to implement theiroccupational choice (Sampson & Peterson, 1984, p. 1).

These criteria served as heuristics for the development of ageneric CACG Evaluation Instrument.

An item pool was developed by the authors to measure each ofthe six criteria described above as well as to measure generalimpressions and human factors (i.e., user friendliness). TheinAial instrument consisted of seven content scales, one foreach objective and one for the computer effect. Followingexternal reviews of items by a variety of career ci-uidanceprofessionals, including the developers of DISCOVER, SIGI andSIGI PLUS, and subsequent editing, a total of 64 items wereretained fc7 field testing. A five-point Likert-type ratingscale was adopted where 1 = strongly disagree, 2 = disagree, 3 =neutral, 4 .E agree, and 5 m strongly agree. Three parallel formsof the E4-item questionnaire were developed, one for each system(Peterson, Sampson, & Reardon, 1985).

7

Page 8: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

4

Xothodology

BubiectsTwo cohort groups of students, one for each successive

semester from an introductory psychology course, were combined torender a pool of 132 subjects. The subjects in the first cohortwere randomly assigned to DISCOVER (nm37) and SIGI (nm31), whilesubjects in the second cohort were randomly assigned to SIGI(nm33) and SIGI PLUS (nm33). Thus, when combined, the DISCOVERgroup consisted of 37 subjects; SIGI, 64 subjects; and SIGI PLUS,33 subjects. There were no significant differences among thethree groups according to age, race, sex, year in school,vocational identity (MVS scores) and career decidedness (OAQscores). Therefore, even though the groups were not randomlydrawn from a single population, it was concluded that the groupswere similar and could be combined to compare systems. Thesubjects elected to participate in the present study from amongother alternatives to meet a course research participationrequirement.

The mean age of the subjects was 18.8 years (50=1.7) with70% being female and 70% white, 11% Native American, 11% black,8% other. The majority of subjects were freshman (64%) and theirdeclared majors were business (33%), psychology (9%), biologicalscience (4.5 %), communications (4.5%), clothing and textiles(4.5%), and nursing (4.5%). Ten percent were undecided. Somesubjects reported having received prior career assistance:individual counseling (25%); career course (11%); and some typeof CACG system (14%).

Computer-Assisted ;armor Guidance Systms=MOM for Schools WISCOVERI. DISCOVER (American College

Testing Program, 1984) is designed to increase decision-makingskills, vocational maturity, specification of career plans, offerinformation about occupations and educational institutions, andincrease the user's self-knowledge concerning interests,abilities, and values (Maze and Cummings, 1982). The fourmodules of DISCOVER include: 1) self-assessment, 2) structuredsearch of occupational alternatives, 3) presentation ofoccupational information, and 4) structured search of educationalalternatives and presentation of educational information.

SIG1(Educational Testing Service, 1984) helps students to examinetheir values, identify and explore options, receive and interpretrelevant occupational data, and master strategies for makinginformed and rational career decisions (Katz, 1973; 1980). Thefive SIGI subsystems include: 1) VALUES, 2) LOCATE, 3) COMPARE,4) PLANNING, and 5) STRATEGY. SIGI PLUS was developed inresponse to comments from users and counselors concerning theeffectiveness of SIGI with a diverse group of individuals,especially adults.

Page 9: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

5

BIM PLUS. SIGI PLUS (Educational Testing Service, 1985) isalso designed to facilitate rational career decision making. Incomparison with SII, SIGI PLUS: 1) providus greater diversity ofself-assessment options, 2) is more flexible in terms of usercontrol of system functioning, 3) provides specific contentmaterial related to the needs of typical adult learners as wellas traditional college-age students, 4) includes content relatedto the job search process, 5) provides for easier customizationof local data, and 6) makes use of color graphics. SIGI PLUS hasnine sections that include: INTRODUCTION, SELF-ASSESSMENT,SEARCH, INFORMATION, SKILLS, PREPARING, COPING, DECIDING, andNEXT STEPS. Katz (1984) provided a description of the initialdesign of the system. The basic assumptions and design featuresof the system are described by Norris, Shatkin, Schott, & Bennett(1985).

ProceduresThe subjects reported at a pre-assigned time to the

university career resource center. Upon reporting, they wereassigned to one of the three CACG systems, and were given a briefoverview of the study. They were asked to complete a researchparticipation release form (see Appendix F) and a demographicquestionnaire (Reardon, 1984b). Upon being given an introductionto the purpose, operation, and procedures associated withDISCOVER, SIGI, or SIGI PLUS, subjects were presented with anexplanation of data collection procedures, and given tour ofthe career resource center. Subjects were then scheduled forinitial two hour appointments to begin work on the assignedsystem. The subjects were told to complete the five SIGImodules, the first three DISCOVER modules, and whichever SIGIPLUS modules most relevant to their career situation. All threegroups were encouraged to ask questions, obtain feedback, andseek support from available staff members during the time theywere in the career resource center using DISCOVER, SIGI, or SIGIPLUS. All systems were completed by the subjects within a tenday period at which time the DISCOVER Progress Record and theDIEM= Evaluation Form, the SIGI Progress Record and $IGIEvaluation form, or the SIGI PLU$ Progress Record and SIGI PLUSEvaluation Dam were completed. Subjects in this study alsocompleted the Comvuter-Assisted Career Guidance Evaluation Form,My Voqational Situation (MVS) by Holland, Daiger, & Power(1980a), and the Occupational Alternatives Question (OAQ) (Zener& Schnuelle, 1972). Subjects were debriefed as a group at thefinal data collection meeting.

Inlitrusontatio4A field test version of the Comoutermhssisted Career

Guidance Evaluation Form consisted of 64 items developed toassess the five objectives included in the evaluation standard.Through principle factoring with varimax rotation, the item poolwas reduced to 24 items which load on six orthogonal factors,each with eigen values greater than 1.0. The factors shown inTable 1 were labeled in descending order of eigen values:Attractiveness of CACG Systems; II, Needs for Occupational

Page 10: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

6

Knowledge; III, Credibility of Alternatives; IV, Knowledge ofOccupational Rewards and Demands; Vf Satisfaction ofAlternatives; and VI, Clarifying Self Knowledge. The final 24-item instrument is shown in Appendix A.

Place Table 1 about heree

The items comprising the above six factors were logicallycombined to form three higher order composite scales: Analysis,Synthesis, and Computer Effect. The Analysis Scale consisted of10 items comprising Factors II, IV and VI and measured how wellthe CACG system helped individuals acquire self-knowledge andoccupational knowledge. Such constructs are fundamental to theability to formulate plausible career alternatives. TheSynthesis Scale was composed of 5 items loading on Factors IIIand V. This scale assessed the degree to which a CACG systemhelped users to identify potential career alternatives. Thethird scale, Computer Effect, was composed of only a singlefactor (I) with 9 items which measured the degree to whichindividuals found interacting with the computer rewarding. Theintercorrelations among the three composite scales ranged from.39 to .60, while the respective alpha reliabilities wereAnalysis, .83; Synthesis, .77; and Computer Effect, .87 (Table2). Thus the scales were considered as independent and reliablemeasures.

Place Table 2 about here

These three scales can now be used to compare the degree towhich different CACG systems perform three vital functions incareer decision-making. These include, becoming familiar withoneself and the world of work (Analysis), developing andevaluating career options (Synthesis), and believing that one isbeing helped (Computer Effect).

The client characteristics that may bear on the impact ofCACG systems were measured by My Vocational Situation (Holland,Daiger, & Power, 1980a) and the Occupational Alternativesf011efftiVon (Zener, & Schnuelle, 1972). The OccuoationmaAlternativge Ovestion (OAQ) consisted of two parts: (a) "list allthe occupations you are considering right now," and (b) "whichoccupation is your first choice? (if undecided, writeundecided)". The test-retest reliability of a questionnaire thatincluded this question was .93 (Redmond, 1973). Two studies(Slaney, 1980; Slaney, Stafford, & Russell, 1981) demonstratedthat the OAQ had considerable concurrent validity with othermeasures of career indecision when the responses were scored asfollows: 1 m a first choice is listed without any alternatives,2 m a first choice is listed along with alternatives, 3 m nofirst choice is listed, just alternatives, and 4 m neither first

10

Page 11: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

7

choice nor alternatives are listed. This scoring system was usedin the present study.

The My Vocational Utuation (MVS) (Holland, Daiger, & Power,1980a) contains three scales: Identity (I), Information Needs(MVSIN), and Barriers (MVSRAR). The Identity subscale of the MVSwas developed by zombining two earlier scales, the VocationalDecision-Making Difficulty Scale (VDMN; Holland & Holland, 1977)and the Identity Scale (Holland, Gottfredson, & Nafzinger, 1975).Factor analyses indicated that these two scales had similarfactor structures and measured the same dimensions for both sexes(Holland, Daiger, & Power, 1980b). The estimate of reliability(KR-20) for college students was .89 (Holland, Magoon, & Spokane,1981). KR 20/s show relatively low external consistency for theMVSIN (male = .79, female = .77) and MVSBAR (male = .45, female =.65), indicating that they resemble checklists more than scales(Holland, Daiger, and Power (1980a). Additional data on thedevelopment and the concurrent validity of the VocationalIdentity scale were presented in Holland, Daiger, and Power(1980b).

For the field test, the DIEM= FrogKess Record, the SIGIErogress Rpcord, and the §IGI PLUS Progress Record (Reardon,1984a) (see Appendices B, CI & D) were designed to ver4fy theextent to which DISCOVER, SIGI, and SIGI PLUS were actually usedby subjects. Basic demographic data and information related tosubjects/ prior exparience with career counseling servicee,including computer applications, were also collected (Reardon,1984b) (see Appendix E) in the field test.

poi' CodingFor data analysis, the OAQ score values were recoded (1) low

and (2) high career decidedness based on the median split of thescores of the subjects in this study. High career decidednessincluded those individuals who indicated either a first choiceonly s2r, a first choice plus alternatives. Low career decidednessincluded those subjects who listed alternatives but no firstchoice, as well as those who had neither a first choice noralternatives.

Subjects/ scores on the Vocational Identity subscale of MyVocational_Situation were similarly divided into two levels ofvocational identity based on the median split of the currentsample. Thus subjects scoring ten or less were regarded ashaving low vocational identity, whereas clients scoring fromeleven to eighteen points were regarded as having high vocationalidentity. The means and standard deviations were reported as11.25 and 4.14 for college men and 10.13 and 4.23 for collegewomen (Holland, Daiger, & Power, 1980b).

The sum of the "N" responses to the four items on theInformation Needs subscale (MVSIN) of the My Vocational Situationprovided an index of subjects/ expressed information needs. Foranalyses, subjects were divided into those who (1) expressed no

11

Page 12: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

8

need for information, and (2) those who expressed current needsfor information. Similarly, the Barriers subscale of the MVS(MVSBAR), provided an index of subjects' perceived barriers inachieving career goals. Subjects were divided into two groups,those who (1) expressed no difficulties, and (2) those who sawbarriers in accomplishing their goals. Subjects' year in school(YEAR) was obtained from the demographic questionnaire andrecoded for analysis into (1) Freshman, or (2) Sophomore, Junior,or Senior.

pats AnalysisA one-way MANOVA with three levels of computer (DISCOVER,

SIGI, or SIGI PLUS) was used to ascertain whether there was amultivariate effect among the respective CACG systems. Thedependent variables included Analysis, Synthesis, aisd ComputerEffect, while age, OAQ scores, and MVS scores were used ancovariates to partial out variance attributed to subject entrycharacteristics. A series of 15 post-hoc 2X3 ANOVA's (two levelsof client characteristics X three levels of computer) wereconducted to determine whether high or low age, OAQ, or MVSscores were related to perceptions of CACG system effectiveness(i.e., analysis, synthesis, and computer effect) among the threesystems.

Results

Zero order correlations among the variables in the study arepresented in Table 3. For these 127 subjects, career decidedness(OAQ), but not vocational identity (MVSID), was significantly (p< .05) related to subjects' perceptions of the cACG systems.Subjects who were more decided about their career goals viewedCACG systems more positively in terms of: (1) helping individualsto acquire self-knowledge and occupational knowledge (Analysis);(2) helping users to identify potential career options(Synthesis); and (3) obtaining a more enjoyable and rewardingcomputer interaction (Effects).

Place Table 3 about here

As was expected on the basis of previous research, careerdecidedness (OAQ) was significantly (p < .001) related to the MVSvocational identity scale. Persons whose career goals are moredecided have a mcre crystallized vocational identity. VocationalIdentity score was also positively correlated to expressed needfor help In diverse areas of concern (MVSIN). There was asignificant positive correlation between information needs(MVSIN) and subjects' perception of the degree to which the CACGsystem was helpful in identifying career options (Synthesis).

The results of a Multivariate Analysis of Variance withthree dependent variables, Analysis, Synthesis and ComputerEffect, demonstrated that there was a significant difference (p <

12

Page 13: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

9

.05) among the CACG systems (see Table 4). The potential sourcesof error due to indecision (4 variables) and year in school wereremoved through covariance.

Place Table 4 about here

Univariate tests of the dependent variables indicated thatthere were significant differences among the three systemsaccording to the Synthesis Scale (p < .02), but not acconding toeither the Analysis or Computer Effect Scales. SIGI PLUSoutperformed DISCOVER, which in turn outperformed SIGI on thedegree to which users were satisfied with the career alternativesgenerated by the systems. Mean ratings of all three scales amongall three instruments were positive.

In order to abzertain whether the effectiveness of the CACGsystem was a function of a client's state of career decidedness,a series of 3 X 2 (Type of Treatment X Level of Attribute) ANOVAswere conducted using high and low groups for year in school, OAQ,and MVS-Identity, MVS-Information Needs, and MVS-Barriers. Therewere no main effects nor interaction on any of the threedependent measures using the year in school, MVS-Identity 0: MVS-Barriers scales as moderator variables. Using the MVS-Information Needs scale as a moderator variable, subjects withinformation needs rated all three systems significantly higher(P < .001) on the Synthesis Scale than subjects with noinformation needs (see Table 5, Figure 1). These results suggestthat individuals with information needs rate all systems higherregarding the generation of alternatives than individuals who donot express information needs.

Place Table 5 and Figure 1 about here

Using the OAQ as moderator variable, there were inconclusiveresults pertaining to the relationship between career decidednessand ratings of computer effectiveness. The ANOVA's revealed nonew information beyond the correlational analyses above.

Discussion

The results of this study are important for three reasons.First, an instrument has been developed to evaluate theeffectiveness of CACG systems. This instrument, the Computer-Assisted Career Guidance EvaluOti9/179=, contains 24-itemscomprising six factors, that were combined to form three higherorder composite scales: Analysis, Synthesis and Computer Effect.The alpha reliabilities of these three scales were judgedsufficiently high to consider the scales reliable measures foruse in comparative research on CACG systems.

13

Page 14: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

10

Second, the Computer-Assisted Career Guidance EvaluationE2XM was subsequently used to compare the effectiveness ofDISCOVER, SIGI, and SIGI PLUS. All three CACG systems ware ratedpositively by most subjects on all three dimensions, Analysis,Synthesis, and Computer Effect. There may be statisticallysignificant differences among CACG systems in each aspect ofperceived effectiveness, but whether these differences are ofpractical significance is still open to question. Individualswho admit to having career informati,n needs respond morefavorably to the career options developed through theirinteraction with the computer than those who do not, regardlessof the system. Those clients who expressed 112 information needs,while less satisfied with alternatives generated by the computerthan those who do, still rated the CACG systems positively onAnalysis and Computer Effect. They ostensibly enjoyed self-exploration and using the computer as strongly as those subjectswith expressed information needs.

Third, these results show that perceived effectiveness ofCACG systems may be related to the state of client careerdecidedness and vocational identity. Individuals who had highcareer decidedness, as measured by the On, differedsignificantly from those who had low career decidedness in theirpreference for a CACG system. This effect, while statisticallysignificant (p < .05) may not be practically significant withcorrelation coefficients between .17 and .20.

These findings suggest that, among the three CACG systemscompared, there are no differences among the systems pertainingto their capabilities for fostering self-exploration, explorationof career options, and the perception that the CACG system washelpful. The subjects using SIGI PLUS rated this system higherin terms of satisfaction with career alternatives generated thansubjects who used DISCOVER or SIGI. A limitation of the studywas that the clients were solicited for career assistance.Further investigations using actual client populations would bewarranted. Caution is also recommended in ieneralizing thefindings of this study to other student populations, particularlynon-white, non-female groups, given the preponderance (70%) ofwhite, female subjects in the current sample.

14

Page 15: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

11

References

American College Testing Program. (1984), DISCOVER for Schools.Hunt Valley, HD: Author.

Cairo, P.C. (1983). Evaluating the effects of computer-assistedcounseling systems: A selective review. The Counselingpsychplopgist, li(4), 55-59.

Chapman, W. (1975). Counselor's handbook for SIGI. Princeton,NJ: Educational Testing Service.

Clyde, J.S. (1979). Computerized career information andguidance systelm. Columbus, OH: The Ohio State University,ERIC Clecringhouse on Adult, Career, and Vocational Education.(ERIC Document Reproduction Service No. ED 179 764).

EducationalGuidance

EducationalAuthor.

Testing Service. (1984). System Of Interactiveand Inforkatign. Princeton, NJ: Author.

Testing Service. (1985). SIGI PLUS. Princeton, NJ:

Fretz, B.R. (1981). Evaluating the effectiveness of careerinterventions. Journal of Counseling Psychology, 21, 77-90.

Fretz, B.R., & Leong, F.T.L. (1982). Career development statusas a predictor of career intervention outcomes. Journal ofCounseling Psychology, 22, 388-393.

Gelatt, H.B. (1962). Decision-making: A conceptual frame ofreference for counseling. Journal 21 Counseling Psychology, 2,240-245.

Harris-Lowlsbey, J. (1983a). A historical perspective. In C.Johnson (Ed.), Nicrocomuters And tho school counselor (pp. 1-16). Alexandria, VA: Americpn School Counselor Association.

Harris-Bowlsbey, J. (1983b). The computer and the decider. TheCounseling psychologist, lip 9-14.

Harris-Bowlsbey, J. (1985). Systematic career guidance andcomputer-based systems. In Z.B. Leibowitz, & H.D. Leo (Eds.).(1985). Adult career development: Concepts4 issues andpractices (pp. 95-107). Alexandria, VA: American Associationfor Counseling and Development.

Holland, J.L., Daiger, D.C., & Power, P.C. (1980a). MyVocationaLSituation. Palo Alto, CA: Consulting PsycholovitsPress.

15

Page 16: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

12

Holland, J.L.1 Daiger, D.C., & Power, P.G. (1980b). Somediagnostic scales for research in decision making andpersonality: Identity, inform..tion, and barriers. journal ofPersonality and Social Psych042901.1 22(6), 1191-1200.

Holland, J.L., Gottfredson, G.D., & Nafzinger, D.H. (1975).Testing the validity of some theoretical signs of vocationaldecision-making ability. Journal of Counseling _Psychology,22, 411-422.

Holland, J.L., & Holland, J.E. (1977). Vocational indecision:More evidence and speculation. JDurnal of Counselingpsychology, Zip 404-414.

Kapes, J.T., Borman, C.A., Garcia, G.,Jr., & Compton, J.W.(1985, April). Evaluation of microcomputer based careerguidance systems with college students: SIGI and DISCOVER.Paper presented at the annual meeting of the AmericanEducational Research Association, Chicago.

Kapes, J.T., Borman, C.A., & Kimberly, R.M. (1985, January).Using microcomputer based career guidance systems in auniversity setting: Reaction data from users. Paperpresented at the annual meeting of the Southwest EducationalResearch Associati,n, Austin.

Katz, M.R. (1966). A model of guidance for career decision-making. Vocational Guidance Journal, 11, 2-10.

Katz, M.R. (1973). Career decision-making: A computer-basedSystem of Interactive Guidance and Informatign ISIGI). FicmProceedings of the 1973 Invitational Conference on TestingProblems--Measurement for Self-Understanding and PersonalDevelopment, Educational Testing Service.

Katz, M.R. (1980). SIGI: An interactive aid to careerdecision-making. Journal of College Student Personnel, al,34-40.

Katz, M. R. (1984). Computer-assisted guidance: A walk-throughwith running comments. Journal of Counselinl and Development,A2, 153-157.

KatL1371.3ta :utttltln, L. (1983). Characteristics of computer-e Counseling Psychologist, 11, 15-31.

Maze, M., & Cummings, R. (1982).Maze, & R. Cummings (Eds.), H2Nguidance eyetem (pp. 97-107).Wisconsin, Wisconsin Vocational

1=7:::asIlls144Madison, WI: University ofStudies Center.

16

Page 17: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

13

Norris, L., & Shatkin, L., Schott, P. S., & Bennett, M. F.(1985). 4,IGX pwsi poyelopaqa and tield test of the computer-based System of Interactive Guidance and Information...PLUSnag. Unpublished manuscript, Educational Testing Service,Princeton, NJ.

Osipow, S.H., Carney, C.G., Winer, J.L., Yanico, B.J., &Koschier, M. (1976). The CAreer Decision Scale. Columbus,OH: Marathon Consulting & Press.

Parish, P.A., Rosenberg, H., & Wilkinson, L. (1979). Careerinformation resources, applications. And research 1950-1919.Boulder, CO: University of Coloradd.

Peterson, G.W., Sampson, J.P., Jr., & Reardon, R.C. (1985).DIEOUR Valuation Egrm & SIG1 Evaluation arm & $IGI PLUSEvaluation Egrm. Unpublished manuscripts, Florida StateUniversity, Project LEARN--Phase II, Tallahassee.

Rayman, ,LR., Bernard, C.B., Holland, J.L., & Barnett, D.C.(1983). The effects of a career course and some other populartreatments. Journal of Vocational Behavior, 22, 346-355.

Reardon, R.C. (1984a). DISCOVER Progress Record, & 51GI ProgressRecord 112g. Unpublished manuscripts, Florida State University,Project LEARN--Phase II, Tallahassee.

Reardon, R.C. (1984b). Computer-Assiste0 Career GgiOanceOuestionnaire. Unpublished manuscript, Florida StateUniversity, Project LEARN--Phase II, Tallahassee.

Redmond, R.E. (1973). Increasing vocational information seekingbehaviors of high school students (Doctoral dissertation,Utiversity of Maryland). Dissertation AbstractsInternational, 24, 2311A-2312A. (University Microfilms No.73-17,046)

Ramer, P., O'Neill, C.D., & Gohs, D.E. (1984). Multiple outcomeevaluation of a life-career-development course. Journal ofCounseling Psychology, 21,1 532-540.

Sampson, J.P., Jr., McMahon, B.T., & Burkhead, E.J. (1985).Using computers for career exploration and decision-making invocational rehabilitation. Rehabilitation CounselingBulletin, 21, 242-261.

Sampson, J.P., Jr., & Peterson, G.W. (1984). Evaluation standar4:Computer-Assisted career guidance systems. Unpublishedmanuscript, Florida State University, Project LEARN--Phase II,Tallahassee.

Slaney, R.B. (1980). Expressed vocational choice and vocationalindecision. Journal of Counseling Psychology, 22, 122-129.

17

Page 18: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

14

Slaney, R.B. (1983). Influence of career indecision ontreatments exploring the vocational interests of collegewomen. Journal of Counseling Psychology, U30, 55-63.

Slaney, R.B.1 & Dickson, R.D. (1985). Career exploration withreentry women. Journal of Counseling Psychology, ;La, 355-362.

Slaney, R.B., Stafford, ICJ., & Russell, J.E.A. (1981). Careerindecision in adult women: A comparative and descriptivestudy. gournal of Vocational Behavior, 11, 335-345.

Splete, H.H. (1984). Computerized career guidance systems andcareer counseling services. Unpublished manuscript, OaklandUniversity, Adult Career Counseling Center, Rochester, MI.

Splete, H.H., Elliot, B.J., & Borders L.D. (1985). Computer-assisted cbreer Amidance systems and career counselingservices. Unpublished manuscript, Oakland University, Adultcareer Counseling Center, Rochester.

Spokane, A., & Oliver, L. (1983). The outcomes of vocationalintervention. In W.B. Walsh and Osipow (Eds.), Handbook pfvocational Dsvchology (vol. 2) (pp. 96-136). Hillsdale, NJ:Lawrence Erlbaum Associates.

Super, D.E. (1973). Computers in support of vocationaldevelopment and counseling. In H. Borow (Ed.), Careerguidance for A pew age (pp. 285-316). Boston: HoughtonMifflin Co.

Zener, T.B., & Schnuelle, L. (1972). An evaluation of the Self-Directed Search (Research Report 124). Baltimore, MD: JohnHopkins University Center for Social Organization of Schools.(ERIC Document Reproduction Service No. ED 061458).

18

Page 19: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

TABLE 1 15

Varimax Rotated Factor Matrix for Student Perceptions of CACG systems

FACTQRSF1 12 F3 14 15

ANALYSIS J (Self Information)

10. The computer was helpful inaccurately clarifying my interests. .370 .123 .139 .102 .053

9. The computer was helpful inaccurately clarifying my values. .166 .314 .382 .195 -.091

3. The computer was helpful in showing mewhether or not I needed more informationabout myself beforemaking career decisions. .251 .501 .145 .034 .133

ANALYpIS II (Occupational Information)

22. The computer helped me to learn muchmore about severaloccupations. .252 .699 .098 .216 .074

23. The computer helped me better understandhow the world of work isorganized. .154 ,566 .000 .026 -.093

18. The computer helped me to become morefamiliar with the educationalrequirements of potentialoccupational choices. .079 .563 -.028 .430 .166

/6

.7Q9

.525

. 243

-.097

.067

.130(table cpntiques)

20

Page 20: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

16

F1

30. The computer helped me to identifyimportant milestones to achieve inattaining a career, such as educationaldegrees,training, orlicenses. .255

5. The computer was helpful in showing mewhether or not I needed more informationabout occupationsbefore making career decisions. .100

19. The computer helpsd me understand thedemands associated with potentialoccupational choices, such as amount offree time, vacations, and continuingeducation. .135

20. The computer helped me to understand therewards potential occupations offer,such as salary, interesting work,prestige,variety, and challenge. .083

SYNTHESIS (Occupational Options)

13. The computer satisfied me with thevariety of career options it gaveme to consider.

21

.307

FAcTORSF2 F 3 F5 F6

.026 .199 .208 -.061

.5;4 .038 .083 .048 .133

.245 .097 .919 .020 .085

.213 .016 au .327 .129

.126 .241 .115 478 .023(table continues)

Page 21: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

17

FACTORSF2 F3 F4 F5 F6

17. I can seriously consider most of theoccupations the computersuggested. .160 -.052 4.05. .058 .142 .090

14. The computer satisfied me with thenumber of career options it gave meto consider. .229 .080 .286 .192 156 -.035

15. The computer presented logical careeroptions given my values,interests and abilities. .090 -.035 524 .029 .289 .155

27. The computer helped me feel confidentthat I would find most of the final listof potentialoccupations satisfying. .292 .188 jili .004 .040 .046

EFFCTS OF THE compuTu

1. The computer helped me become moreconfident of being able to choose asatisfying occupation. .1.522 .138 .425 .011 .142 .037

48. 1 understand myself better now. .583 .298 .064 -.020 .197 .229

59. Using the computer is like talkingwith a career counselor. .549 .171 .090 -.055 .179 -.009

(talble continues)

23 24

Page 22: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

18

F2 F3FACTORSF4 F5 F6Fl

41. The computer helped me to feel moretnpeful of finding a satisfyingcoccupation. .577 .270 .337 .109 .258 .056

55. I have learned about some neweducational programs as a result ofusing the computer. .494 .242 .024 .207 .052 .147

31. I felt the computer understood mycareer problems. .635 .111 .124 .147 -.007 .170

43. The computer answered most of my careerquestions to mysatisfaction. .539 .167 .364 .398 .052 -.174

54. 1 felt better about my career after1 used the computer. .573 .193 .290 .206 .107 .165

51. My family or friends liked theoutcomes suggested by the computer. .601 .133 .179 .037 .147 .143

2526

Page 23: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

19

TABLE

Intercorrelations Among Scales pf the Computsr-Assisted

v

Standard Diviatigns 4nd Alpha Reliability Coefficients

Ingc127)

Measure

CorrelationsGrand

1 3 Means SD Alpha

1. Analysis(10 items) 1.0 .777 .487 .83

2. Synthesis(5 items) .39 1.0 .606 .714 .77

3. Effective-ness .60 .58 1.0 .437 .630 .87(9 items)

27

Page 24: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

20

Table 3

Inttmarglatima_HAtrix (nte127)

Variables 1 2

gorrelaltions7 83 4 5 6

1.

2.

3.

4.

5.

6.

7.

8.

Analysis

Synthesis

Effect

Yeara

OAQb

MVSIDC

MVSINd

MVSBARe

.39ft**-

.02 .08

.17 * .18*

.05 .10

-.06 .26**

-.06 -.02

-

-.02

.20*

.04

.05

-.08

-

-.11

.02

.12

.00

0.

.29***

.05

-.01 -.10

/10D

-.05

a Year in school (122Freshman, 2inSophomore, Junior, Senior)Occupational Alternati7es Question Score (limfirst choice only gx firstchoice plus alternatives, 2alternatives only 2r neither firstchoice nor alternatives)

My Vocational Situation - Vocational Identity Score (lirlow identity,2ohigh identity)

My Vocational Situation - Information Needs Score (ligno informationneeds, 2*need information)

My Vocational Situation - Barriers Scote (1urno barriers, Zwbarrierspresent)

* g < .05** p .01*** p < .001

28

Page 25: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

21Table 4

NANOVA Summary Table

SIGI (n=60) DISCOVER (n=33) SIGI PLUS (n=29)Mnxttnowowq....PIL.npnwtRwxnwnnn.nJI1lLwttw§11,.....+......IILP..M.PWIPPP.Ea

DermdentVariables

Analysisa .76Synthesisa.53Effect& .37

ModeratorVariables

Year" 1.55OAQc 2.30MVSIDd 10.50MVSINa

f1.02

MVSBAR 3.57

.49 , .81 .45 .7R .55

.74 .64 .83 .72 .56

.65 .54 .68 .49 .54

.87 1.45 .75 1.62 .90

.56 2.42 .56 2.34 .484.50 11.33 5.24 9.76 4.901.19 1.52 1.44 .62 .78.62 3.39 .79 3.59 .68

Multivariate Tests of Significance

Test Name Value F DF Error DF Sign. of F

Pillais .22 1.82 15 342.00 03*Hotellings .25 1.83 15 332.00 03*Wilk's .79 1.83 15 309.58 .03*Roys .14 .05*

Univariate F Tests (5,114)

Vari.able SS MS F Sign of F

Analysis 1.50 .30 1.23 .30Synthesis 6.74 1.35 2.73 .02*Effect 2.57 .51 1.30 .27

a Scoring: 5-point Likert-type scale, where -2 = strongly disagree -I =

disagree; 0 = neutral; +I =agree; +2 = strongly agreeb Scorings I = freshman; 2 = Sophomore; 3 = Junior; 4 w Senior; 5 = Graduate

Student; 6 = Adult, not presently tlenrolledScoring: 1 = first choice only, no alternatives; 2 = first choice withalternatives; 3 no first choice, alternatives only; 4 = neither firstnor alternatives

Scoring: total number of "False" responses on the Vocational Identitysubscale

Scoring: total number of "No" responses to the four items identifyingcurrent information needs

f Scoring: total number of "No" responses to the four items identifyingcurrent barriers to meeting career goals

* p < .05

29

Page 26: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

22

Table 5

Comparison eg the 4ya1spis Function According U) CACGsystems with My Vocational 4itgAti9n Information Needs Scaleas Moderator Variable

Systems

MVSIN SIGI DISCOVER SIGI PLUS

M =HIGH SD =

(N =*Ns\

M =LOW SD =

(N =

.63a M = .88 M = .83

.72 SD = .59 SD = .4635) (N = 22) (N = 13)

.27 M = .15 M = .66

.83 SD =1.03 SD - .6231) (N = 11) (N = 17)

Source ofVariation

RiIii-iffiaiSystemMVSIN

SystGmX MVSIN

Explained

Residual

SS

Source Table

MS DFSign. of

5.79 1.93 3 3.99.96 .48 2 .99 .37

5.12 5.12 1 10.59 .001***

1.12 .56 2 1.16 .32

6.91 1.38 5 2.86 .02*

57.01 .48 118

a On a 5-point Likezt-type scale scored as follows: -2 m. stronglydisagree, -1 Is disagree, 0 Ja neutral, +1 agree, and +2 m. strongly

agree.

* p < .05** p < .01*** p < .001

Page 27: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

Figure 1

Comparison of the Synthesis FunctionAccording to CACC Systems with MVSINas Moderator Variable

1 .0

0.3

0.3

0.4

c.:i

t.t.;

SIGI

C) High

Low

oVU SIG.! PLUS

CA

31

Page 28: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

Name

l4APPENDIX A

COMPUTER-ASSISTED CAREER GUIDANCE EVALUATION FORK'

Date

Soc. Sec. No.

1

PLEASE USE THIS FORM TO EVALUATE THE SYSTEM.

1. Identify the sections/modules that you are evaluating today.

2. List all occurations you are considering right now.

3. Which occupation is your first choice? (If undecided,write "undecided.")

FOR QUESTIONS 4 AND 51 CHOOSE A RESPONSE AND PLACE THE NUMBER IN THESPACE IN THE RIGHT MARGIN WHERE INDICATED:

4. How well satisfied are you with your first choice?....4.1. Well satisfied with choice2. Satisfied, but have a few doubts3. Not sure4. Dissatisfied, but intend to remain5. Very dissatisfied and intend to change6. Undecided about my future career

5. How long did you use the computer at this session?...1. 30 minutes or less2. 30 minutes to 1 hour3. 1 hour to 1 1/2 hours4. 1 1/2 hours to 2 hours5. 2 hours or longer (Continued on next page)

1 Authored by Gary W. Peterson, Ph.D., Rebecca E. Ryan-Jones, N.Ed.,James P. Sampson, Jr., Ph.D. and Robert C. Reardon, Ph.D. Center forthe Study of Technology in Counseling and Career Development, FloridaState University. April, 1988.

32

Page 29: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

PLEASE ANSWER THE FOLLOWING QUESTIONS BY CIRCLING THE APPROPRIATENUMBER ACCORDING TO THE KEY BELOW.

1 2 3 4

Strongly Disagree Neutral AgreeDisagree

(SD) (D) (N) (A)

1. The computer helped me to learnmuch more about severaloccupations.

2. The computer was helpful in showingme whether I needed more informationabout occupations before making careerdecisions.

3. Using the computer was like talkingto a career counselor.

4. The computer presented logicalcareer options given my values,interests, and abilities.

5. The computer helped me to understandthe rewards potential occupationsoffer, such as salary, interesting work,prestige, variety, and challenge.

6. I felt the computer understood mycareer problems.

7. I have learned about some neweducational programs as a result ofusing the computer.

8. The computer helped me feta.confident that I would find most ofthe final list of potentialoccupations satisfying.

9. The computer satisfied me with thevariety of career options it gaveme to consider.

10. The computer helped me to becomemore familiar with the educationalrequirements of potentialoccupational choices.

11. The computer was helpful inaccurately clarifying my values.

33

25

2

PP

5StronglyAgree(SA)

Q N A

6Does NotApply(DNA)

SA, DIM

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

1 2 3 4 5 6

(Continued on next page)

Page 30: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

26

. 3

1 2 3 4 5 6Strongly Disagree Neutral Agree Strongly Does NotDisagree Agree Apply

(SD) (D) (N) (A) (SA) (DNA)

12. The computer helped me to feel morehopeful of finding a satisfyingoccupation.

13. I can seriously consider most of theoccupations the computer suggested.

14. My family or friends would like theoutcomes suggested by the computer.

15. The computer satisfied me with thenumber of career options it gave meto consider.

16. The computer was helpful in accuratelyclarifying my interests.

17. The computer was helpful in showingme whether I needed more informationabout myself before making careerdecisions.

18. The computer helped me understandthe demands associated with potentialoccupational choices, such asamount of free time, vacations, andcontinuing education.

19. The computer answered most of my careerquestions to my satisfaction.

20. The computer helped me to identifyimportant milestones to achieve inattaining a career, such as educationaldegrees, training, or licenses.

21. The computer helped me better understandhow the world of work is organized.

22. I understand myself better now.

23. I felt better about my career afterI used the computer.

24. The computer helped me become moreconfident of being able to choose asatisfying occupation.

fiD D1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

1 2

4 4 sik DEA3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

3 4 5 6

Page 31: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

27

t

Arretent B

DISCOVER PROGRESS RECORD

(print)

Directions: Curricular Career Information Service (CCIS) is

seeking to evaluate the quality of many of its programs and

services in order to improve them. You can help in two ways: (1)

complete at least some work in Sections 1*, 2*, and 3* of

DISCOVER, and (2) complete the evaluation forms provided. This

set of instruments will help us evaluate DISCOVER. Thank you for

your help.

AppointmentDate:

OM, 4.

N.WOMO. OOmmimmt

Time OnDISCOVER

_hr.

hr.

W104.101000bmwm.

Check SectionUsed:

...min. * ...I. Learning aboutyourselfa. Insterests

..min._b. Abilities...co, Values,

* 2, Searching foroccupations

* ....3. Learning aboutoccupationsa. Browsing

Detail questions.4. Searching for

educationalinstit4tions

aft MID 41=1,0.

Note: If you have completed your use of DISCOVER and do not plan tomake another appointment to use DISCOVER, please turn this page andcomplete the remainder of these evaluation forms.

Page 32: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

Appendix C

NAME:

28

(print)

SIM PROGRESS RECORD

Directions: CurriLmlar Career Information Service (CCIS) isseeking to evaluate the quality of many of its programs andservices in order to improve them. You can help in two ways: (1)complete at least some work in Sections 1 through 5 of SIM, and(2) complete the evaluation forms provided. This set ofinstruments will help us evaluate $IGI. Thank you for your help.

AppointmentDate:

(1) / /Owlme wom4

)

(3)

Time OnSIM

hr.

hr.

hr.

min.

..Check SectionUsed:

1. VALUES

......2. LOCATE

3. COMPARE

PLANNINO

5. STRATEGY~..filmemAlmomoi..WW400~O.0.1wOrMil....06

Note: If you have completed your use of SIGI and do not plan tomake another appointment to use $IGI please turn this page and com-plete the remainder of these evaluation forms.

36

Page 33: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

29

APPENDIX D

NAME:(print)

SIG! PLUS PROGRESS RECORD

Directions: Curricular Career Information Service (CCIS) isseeking to evaluate the quality of many of its programs andservices in order to improve them. You can help in two ways:complete whichever sections of SIGI PLUS are most relevant t,your needs, and (2) complete the evaluation forms provided. Thisset of instruments will help us evaluate SIG! PLUS. Thank youfor your help.

Appointment Time onDate: SIG! PLUS

Check SectionUsed:

hr. min. 1. INTRODUCTION

hr. min. 2. SELF-ASSESSMENT

hr. min. 3. SEARCH

4. INFORMATION

5. SKILLS

6. PREPARING

7. COPING

8. DECIDING

9. NEXT STEPS

Note: If you have completed your use of SIGI PLUS and do notplan to make another appointment to use SIGI PLUS, please turnthis page and complete the remainder of these evaluation forms.

Page 34: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

Appendix E

# COMPUTER-ASSISTED CAREER GUIDANCE QUESTIONNAIRE

Curricular-Career Informaticm Service

Florida State University

Name

MailingAddress

Date

Zi p

O4 MOMMIPMMINWII,MON.MAMO.W.0014.001.1110101.,....10.101.1000WWWW.MMOS

Phone__

Course Prefix Course Number Section Number

CHOOSE A RESPONSE AND PLACE THE NUMBER IN THE SPACE IN THERISHT 11 AWN

1. Major Mee next page)...........

2. Age

3. Sex1. Male 2, Female

4. Ethnic Group1. Black 5. Anglo/White2. Spanish Surnamed 6. Other3. Asian American 7. Prefer not to respond4. Native American

1 0 1.0

2.

3.

4,

5. Year in School 5.1. Freshman 4. Senior2. Sophomore 5. Graduate Student3. Junior 6. Adult not presently enrolled at FSU

Items 6-12 refer to prior experience withcareer counseling services

1. Yes 2. No

6. Individual career counseling 6.

7. Group career counseling 7.

S. Interest/ability/personality assessment O.

9. Career Course 9.

10. Self study career workbook 10.

11. Career Workshop 11.

12. Computer-Assisted Career Guidance System 12.

If yes, what system

VOMMOOWOR.O.

11011.4044.110~WO

WOOMOW.100411.

IMPS..001110.0.1.1.

Page 35: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

IL ACIEMCES

MAJOR?

IDUCATIONStiont/

31

Anthropology 131 Elementary EducationSiological Science 131 &notional Dist./LearningChemistry Disabilities UAW a MR WWIClassical Language 133 teglIsh Education 174 Library ScienceLiterature 134 Evaluation I Must:fomentComputer Science 135 foundations at Education BUMComparative & 131 Health Education 171 MusicVorld Lit. 137 Higher EducationEnglish 131 Instructional Systems MARGeology 137 Leisure Servicoo4 Studies 172 NursingGeophysical Fluid Dyn 140 Mathematics EducationHistory 141 Media Education Ulla MIMIMathematics 142 Mental Retardation 17$ ECOROSitsMedical Technology 143 Movenent Science Education 174 GeographyMeteorology 144 Multilingual/Multicultural 175 Political ScienceModern Longuages Education 174 Public AdministrationMolocular Siophysics 145 Physical Education 177 SociologyOceanography 144 Retding Education 171 Urban 4 Reg. PlanningPhilosophy 147 itehabilitatiun Services t79 Cert. in Public Ada.Physics 141 Science EducationPsycology 147 Social Studies Education EMU WMReligion 150 Special Educatioa

Ito Social WorkStatistics 151 Visual Disabilities

ILL&Susiness

procknort

Audiology I SpeechPathology

Consunications

Criainology

=Ili

SNCINEERING152 Engineering

Una =IOW153 Clothing I Testiles154 Home tinily Lilo135 Home Economics154 Hone Economics Education157 Mirriltge & family151 Nutrition A food Science

UNTERDISCIPLINM15P American StudiesAdult Education160

Comprehensive 141Poe. Ed.

162Coenseling

143Hunan Systems

144Early Childhood Ed. 145Educational Ada.

166147

Ed. Psychology141

Educational Resoarchi Testing

Asian Studies, EastHumanitiesInter-American StudiesInternational AffairsJunior College Instr.Marriage family Lis.Physics Inter. Pr.:straitSlavic & East tut. Stud.Social Sciences

BEST COPY AVAILABLE

:3 9

lag=141 Theatre

=WU Ana112 Art143 Art Education114 Art History115 Dance114 Interior Design

Page 36: WMftee************************************** · Assisted Career Guidance_Evaluation Form, Pi Vocational Situation (MVS) by Holland, Daiger, & Power (1960a), and the Occupational Alternatives

p

Appendix F

RESEARCH PARTICIPATION RELEASE PORN

I give Dr. Robert Reardon and Dr. James Sampson of Florida

State University, permission to examine my responses on various

questionnaires and research instruments teleted to an evaluation

of the DISCOVER and/or SIGI computer-assisted career guidance

systems I understand that at no time will the responses on any

questionnaire or research instrument be identified by name in any_

research report. I further understand that I can have access to

my questionnaires and research.instruments at any time.

NAME (please print)

SIGNATURE DIM

32