Comparing Senior And Sophomore Comparing Senior And Sophomore Knowledge and Confidence Concerning Knowledge and Confidence Concerning Academic AdvisingAcademic Advising

Anecdotal evidence suggested that a discrepancy existed between what faculty believed students knew as a function of

faculty advising and what students actually did know.

We specifically questioned whether students had the knowledge necessary to successfully navigate advising issues on campus.

We wanted empirical evidence to support or refute the perception that students had the necessary knowledge concerning academic

planning, use of academic and career development resources, and the confidence to engage in the tasks necessary to control

and master their own academic endeavors.

We created an assessment to evaluate five specific goals related to advising, recognizing that if students did not demonstrate

proficiency on the assessment, additional programming (such as a peer advising program) would be necessary. Sophomores were

assessed in Spring 2007 and it was found that they were proficient in advising knowledge.

Melynda CottenPeter Swerdzewski

Sara J. Finney Anna Lynn Bell

Methods

Using assessment to determine if students lack the knowledge and confidence that would necessitate a peer advising programUsing assessment to determine if students lack the knowledge and confidence that would necessitate a peer advising program

Results

Sample Multiple-Choice Item:What is the minimum cumulative grade point average a student must maintain to be in good academic standing? 1.0 1.5 2.0 2.5 3.0

Sample Confidence Item:How confident are you in your ability to interpret your degree progress report? No confidence at all A little confidence A fair amount of confidence Much confidence Very much confidence Complete confidence

Sample Attitude Item:To what extent would you trust the information from a student peer adviser? Select the statement that is most characteristic of you. I would trust information from a peer adviser more than I would a faculty adviser. I would trust information from a peer adviser as much (equally) as I would a faculty adviser. I would trust information from a peer adviser somewhat less than I would a faculty adviser. I would not trust at all information from a peer adviser.

Implications and Uses of Results

JAMES MADISON UNIVERSITY

We specifically questioned whether other groups of students, specifically seniors, also had the knowledge necessary to

successfully navigate advising issues on campus. Seniors were chosen to complete the instrument because for many students the

senior year is the most critical year.

We wanted empirical evidence to support or refute the perception that seniors had the necessary knowledge concerning academic planning, use of academic and career development resources, and the confidence to engage in the tasks necessary to control

and master their own academic endeavors.

Seniors were requested to complete the same advising instrument as the previous cohort of sophomores, and the cross-sectional

results from the seniors and sophomores were compared. These results will be used to inform institutional decisions about advising.

Empirical evidence suggests that seniors and sophomores have similarly-high levels of knowledge related to advising; although, not surprisingly, seniors tended to know more than sophomores

with respect to some aspects of advising measured by the instrument.

Stage One: Sophomores’ Knowledge of Advising (Spring 2007)

Stage Two: Seniors’ Knowledge of Advising (Fall 2007)

• The Instrument. An advising instrument was created in Spring 2007 to measure sophomores’ knowledge, skills, and confidence related to academic advising at our institution. This same instrument was administered to seniors to measure their knowledge, skills, and confidence.

• The instrument included items representing five objectives related to different aspects of advising-related knowledge, skills, and confidence.

• Additionally, ancillary questions were included on the instrument to measure whether or not students would use a peer advising center, and the degree to which they would trust a peer advisor.

• Data from the university’s student administration system were used to gather demographic information about students.

• The Administration: Seniors. Seniors were requested via e-mail to complete the Web-based instrument during their own free time at some point in a multi-week window.

• Most students were required to participate in the assessment to fulfill their graduation requirements at the institution.

• Due to a change in policy, many students were subsequently not required to participate in the assessment.

• 1,128 seniors completed the instrument out of approximately 2,700 who were requested to complete the instrument. Data from 956 students were determined to be admissible per data cleaning and outlier analyses, and were thus analyzed.

• The Administration: Sophomores. A random sample of sophomores were required to complete the advising instrument during a university-wide assessment day.

• Students were assigned to take the instrument via a Web-based application in a fully-proctored testing session.

• From a total population of 3,177 sophomores, 401 students were randomly sampled and data from these students were analyzed.

• The Comparison. Data from seniors were compared to data from sophomores who completed the same instrument.

Seniors Sophomores

Overall 77.59% 66.07%

Item 6 – dates and deadlines 94.9% 95.5%

Item 10d – test scores 68.8% 83.3%

Item 10f – advisor contact 95.9% 73.1%

Item 14 – Registrar’s Office 89.5% 84.8%

In general, seniors and sophomores scored similarly on the instrument; however, seniors scored higher on some items. Outcome-level comparisons are listed below along with select item-level findings.

• Four items that address knowledge and locations of resources, including:

• Registration dates and deadlines• Use of e-campus (financial aid, test

scores, advisor contact)• The Registrar’s Office• The Career Planning Office

•Six items that address students’ understanding of their responsibility in academic planning, including:

• Course overrides• Changing majors• Scheduling and preparing for a

meeting with advisor• Legitimate expectations of

advisor (Item 24)• Legitimate expectations of

students (Item 25)

Seniors Sophomores

Overall 68.09% 65.98%

Item 9 – course overrides 90.6% 24.9%

Item 16 – advisor scheduling 89.1% 85.8%

Item 24c – best courses 65.3% 76.8%

Item 24d – contacting parents 92.7% 87.3%

Item 25g – relevance of courses 32.4% 33.9%

Seniors Sophomores

Overall 69.14% 62.06%

Item 4 – minimum GPA 83.3% 70.8%

Item 13c – add/drop timeline 51.4% 57.1%

Item 15c – catalog usage 20.4% 17.7%

•Nine items that address students’ understanding of the policies and processes related to academic planning, including:

• Credit requirements (General Education, graduation, etc.)

• Grade point requirements• Course withdrawal rules• Which academic requirements apply to various situations

Seniors Sophomores

Overall 88.70% 84.29%

• All items were measured on a 6-point response scale from no confidence (1) to complete confidence (6).

• Both seniors and sophomores were most confident at navigating all of the dimensions of e-campus.

• Seniors were least confident in their ability to complete graduation requirements without the help of their advisor.

Seniors Sophomores

Overall 3.99 3.76

Item 17 – without advisor help 3.77 3.53

Item 28 – navigate e-campus 4.33 4.30

Seniors Sophomores

Item 18 – comfort with peer adv. 56.5% 67.1%

Item 19 – would trust peer adv. 29.6% 40.9%

Item 32 – satisfied with advising 67.1% 70.1%

Outcome 1: Student knowledge of academic resources

Outcome 2: Understanding of the student's role in academic advising

Outcome 3: Knowledge of the nuts and bolts of academic advising like how to use e-campus, how to register for classes and how to make a four-year plan

Outcome 4: Awareness of special opportunities like study abroad, internships, and competitive scholarships

Outcome 5: Student confidence in fulfilling graduation requirements and utilizing academic tools and resources without the help of their faculty adviser

Ancillary Analyses

•Only one item was used as an indicator for this outcome. One should be cautious in making inferences from this single item to the overall outcome because the item clearly does not cover the breadth of the outcome.

•Five items that address students’ confidence in the following:

• Completing graduation requirements

• Ability to interpret degree progress report

• Process for requesting an override

• Use of Web site to gather requirement information

• Navigating e-campus (student registration Web portal)

Empirical evidence indicated that students’ knowledge and confidence related to advising at JMU was quite high, despite the lack of peer advising contact the students had. Approximately 67.1% of the senior sample (N =641) and 70.1% of the sophomore sample (N =281) indicated that they are satisfied with the advising they have received at JMU. For those 32.9% (N =315) of seniors and 29.9% (N = 120) of sophomores who indicated they were dissatisfied with advising, primary reasons stated for this lack of satisfaction included the belief that advisers lack competence or quality in the information they provide, and scheduling issues detract from the quality of advising at JMU. Students suggested that advisers become more knowledgeable in advising, and that the scheduling of meetings improve.

Eight items, not tied to learning outcomes, were used to assess students’ comfort with peer advisors, perceived trust of peer advisors, and general satisfaction with the advising at the institution.

It appears that senior and sophomore students have similar levels of knowledge and confidence related to the information and skills necessary to navigate advising-related processes at the institution.

Specifically, seniors and sophomores scored similarly on all five outcomes; however, seniors were slightly higher on most items. Interestingly, when compared to sophomores, seniors reported that they would be less comfortable approaching a peer advisor and would be less trusting of the information from a peer advisor.

Importantly, it was found that both sophomores and seniors were proficient with this information and skill set, thereby negating the need for a peer advising center to provide this information and set of skills.

The average senior in the testing sample… • is 21 years, 7 months old• is female• is Caucasian• was admitted to JMU as a traditional

freshman• began at JMU during the Fall of 2004• is an in-state student• has 118 earned credits• has an SAT score of 1145• is an IDLS major

The average sophomore in the testing sample…• is 19 years, 8 months old• is female• is Caucasian• was admitted to JMU as a traditional

freshman• began at JMU during the Fall of 2005• is an in-state student• has 53 earned credits• had an SAT score of 1148• is a Health Sciences major

Sample Items