internship office/brightspotcdn.byu.edu/53/0c/8a0d... · evaluation project (fall 2016) the 2014...

Internship Office Syllabus Evaluation Project

IMAGE BY MARK A. PHILBRICK [CC BY-SA 3.0 (HTTP://CREATIVECOMMONS.ORG/LICENSES/BY-SA/3.0)], VIA WIKIMEDIA COMMONS

Presenter

Presentation Notes

This study was prompted by the 2014 Internship Office self-study during their unit review. Evaluation was completed by Shiloh Howland (MS candidate in Instructional Psychology and Technology)

Syllabi evaluation project: What it is and what it is not

BY RICHY SCHLEY (OWN WORK) [CC BY-SA 3.0 (HTTP://CREATIVECOMMONS.ORG/LICENSES/BY-SA/3.0)], VIA WIKIMEDIA COMMONS AND PUBLIC DOMAIN,

HTTPS://COMMONS.WIKIMEDIA.ORG/W/INDEX.PHP?CURID=630874

Presenter

Presentation Notes

Different perspectives. Picture on the left is the moon from a good backyard telescope. Picture on the right is moon as seen by the astronauts of Apollo 17. Both views are correct but internship coordinators have a much more detailed view of their courses. I saw your courses just through your syllabi. There may be parts of your course that I could not see, just like I can’t see that boulder from my backyard telescope.

Internship Office should monitor the following aspects of the Internship Policy: Specific course numbers students can enroll in to receive academic credit for internships.

A minimum of 42 hours of work at an internship for every 1 hour of academic credit.

No retroactive credit given to students for internships or work completed in the past

Internship Master Agreements (IMAs) signed and filed with the Internship Office.

A suitable internship experience, defined as one which has “material relevancy to the student’s academic discipline” and which gives the student supervised experiences to help the students develop skills and knowledge.

Direction of students desiring international internships to the Kennedy Center.

Agreement by the internship provider with the learning objectives the student will meet during the internship.

Commitments by the internship provider to monitor and report on student progress and attendance and to assess student learning.

An Internship Oversight Committee in departments or colleges/schools.

A pre-internship orientation at which students learn about registration, assignments and expectations, grading, standards of personal conduct, nature and amount of supervisory contact, university liability insurance, and the IMA.

Procedures for unanticipated termination of an internship.

Routine assessment of internship providers by departments to ensure students are having quality learning experiences.

A syllabus for every internship course that includes purpose or learning outcomes of the course, a course outline, assignments and expectations, test, evaluation, and grading instructions.

Course assignments that are based on combinations or elements of texts/readings, papers/reports, projects/research, and demonstrations or presentations.

IRAMS

IMA (sections 4.1 & 4.3)

Survey

Syllabi evaluation

project (Fall 2016)

Presenter

Presentation Notes

The 2014 self study specified 14 areas that the Internship Office should monitor the compliance of. Nine of these are in some way monitored already (to varying degrees) by IRAMS, the IMA, or the survey we are sending out in March 2017. That left 5 aspects that I went looking for evidence of in the syllabi.

Syllabi evaluation project (Fall 2016) O

vera

ll co

mpl

ianc

e 1 A pre-internship orientation at which

students learn about: • registration using IRAMS • assignments and expectations • grading • standards of personal conduct • nature and amount of supervisory contact • University liability insurance • the IMA • procedures for unanticipated termination of an

internship

2 Routine assessment of internship providers by departments to ensure students are having quality learning experiences.

3 A syllabus for every internship course that includes:

• purpose or learning outcomes of the course • a course outline • assignments and expectations • test, evaluation, and grading instructions

4 Course assignments that are based on combinations or elements of:

• texts/readings • papers/reports • projects/research • demonstrations/presentations

Presenter

Presentation Notes

The 5 aspects are really 4 – because “procedures for unanticipated termination of an internship” is part of the pre-internship orientation. I examined the syllabi for compliance in these 4 areas, as well as overall compliance.

Syllabus collection and rating (Fall 2016) Requested syllabi for 133 internship courses

Received 88 syllabi for 101 courses

32 courses did not submit syllabi

Simplified rubric Fully compliant

Partly compliant

Insufficient evidence to rate

Pre-internship orientation

Assessment of internship providers

Course details

Course assignments

Overall compliance

Presenter

Presentation Notes

Adrienne emailed everyone in September 2016 asking for copies of their internship course syllabi. We received 88 syllabi covering 101 courses and 32 courses did not submit syllabi. Adrienne, Dr. Kristine Hansen, and I discussed what we thought full compliance, partial compliance would look like in syllabus. We examined 7 syllabi together and rated them. We then discussed our ratings to define what full or partial compliance looked like. The term “insufficient evidence to rate” was chosen because just because some aspect is not visible in the syllabus, does not mean it doesn’t exist. It just means we can’t see it. The rubric shown here is conceptual – the actual rubric was more detailed.

Syllabus rating: Individual categories

2

37

20

97

53

30

80

0

46 34

1 4

32 32 32 32

Pre-internship orientation Assessment of internship providers Course details Course assignments

Fully compliant Partly compliant Insufficient evidence to rate Missing

Presenter

Presentation Notes

Results – For those 101 courses, there were varying levels of compliance in those 4 aspects of the internship policy. This is just baseline information and is not a complete view of any course.

Syllabus rating: Overall compliance

1 [VALUE]

[VALUE]

[VALUE]

Fully compliant Nearlycompliant

Partlycompliant

Insufficientevidence torate/missingn = 133

Presenter

Presentation Notes

Looking just at overall compliance – Vast majority were partly compliant which is good but not necessarily helpful to know. Most syllabi were partly compliant – which was about what we expected.

Syllabi scoring (Fall 2016) 1 Pre-internship

orientation Scores for each category:

1 – fully compliant 0.5 – partly compliant 0 – insufficient evidence to rate

Total scores range from 0 to 4

2 Assessment of internship providers

3 Course details

4 Course assignments

Presenter

Presentation Notes

To gain a better understanding of how those partly compliant syllabi, scored each category 0, 0.5, or 1. This would let us see how close to compliance the ‘partly compliant’ syllabi were. This score is the one found in the letters sent out in the last weeks of March 2017.

[VALUE] [VALUE]

31

22 23

16

0 3

[VALUE]

4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0

Syllabus scoring

Score n = 133

Presenter

Presentation Notes

Overall, most syllabi were close to compliance and could become fully compliant with the internship policy with a few changes to the syllabi.

Pre-internship orientation STDEV 496R

EXSC 688R

HLTH 496R

COMMS 399R/496R

LAW 599R

Presenter

Presentation Notes

Examples of how pre-internship orientations are handled by various courses

Assessment of internship providers

IS 599R

PDBIO 399R ELANG 399R

Presenter

Presentation Notes

Examples of how internship providers are assessed by various courses

Course details

HIST 199R

PSYCH 399R/496R

Presenter

Presentation Notes

Examples of course details in various courses. The most commonly missing item in this area was a grading scale.

Course assignments

PSYCH 399R/496R

ENGL 399R

Presenter

Presentation Notes

Examples of interesting course assignments.

Course assignments ITAL 399R

ACC 599R

RECM 496R

Presenter

Presentation Notes

Examples of interesting course assignments.

Dr. Kristine Hansen Syllabus: ENGL 399R

Score: 3 out of 4 possible points

The following 3 slides show how Dr. Hansen made changes (shown in yellow) to her syllabus to bring it in to full compliance with the Internship Policy.

She estimates it took 30-45 minutes to make the changes necessary.

ENGL 399R p. 1

ENGL 399R p. 2

ENGL 399R p. 3

Additional help or questions

If you have questions about this evaluation project, please contact Shiloh Howland at [email protected]

If you would like help developing your syllabus, please contact Dr. Kristine Hansen ([email protected]) or Adrienne Chamberlain ([email protected])

Each internship coordinator will receive a letter with the results of their syllabus(syllabi) evaluation.

A survey regarding the Internship Office will be sent out at the end of March 2017.

mailto:[email protected]



internship office/brightspotcdn.byu.edu/53/0c/8a0d... · evaluation project (fall 2016) the 2014...

Documents