American Accounting Association Annual Meeting Effective Learning Strategies Forum

August 2011 Carol A. Hartley

Providence College chartley @ providence.edu

 

Group Examinations as an Engaged Learning Supplement to Individual Examinations

Introduction (Abstract) High end students typically have mastered the study technique of reviewing material post-exam for any areas in which they experienced a weakness. More typical students may just “dump their minds” of material after an exam and try to put it out of their minds until the next one. Students sometimes complain that they are not certain why their answers may be wrong; some students still do not understand the material after reviewing a graded exam. An effective learning strategy is to use group and individual examination techniques to motivate students to re-study the exam material after the initial individual exam and to provide a subsequent group exam process that engages students in developing better understanding. This technique creates an opportunity for students to earn a grade scale on examinations that present challenging material, e.g., the intermediate accounting sequences, versus faculty providing a simplistic grade scaling technique on difficult exams. End of course survey results in this example from intermediate accounting also show that students perceive that the technique helps them learn the material.

The Examination Conundrum Faculty all want their students to learn, but how do we provide them with incentives to not give up and to go back to the task until they master the material and concept. Accountancy is a progressive study, and difficulty at the earlier stages in a course can be compounded as we move to the even more difficult material. Students may think of examinations as a means to a grade, but beyond basic assessment, how do we as faculty turn examinations into learning opportunities? How do we get students to go back at it once again to self-correct and re-learn the material as a strong foundation for further study?

A group examination on the same material after an individual examination provides an incentive for an improved grade which the students ostensibly want, but it also provides another opportunity for a deeper engagement in the material that students may have misunderstood or did not pay careful enough attention to the first time. Many faculty will do a passive review post-exam, but a more effective active learning technique is to engage the students in a second chance group solution which provides another incentive to review and study what they may have missed. The group examination engages students in deeper discussions of material outside of class, and helps students learn more while actively earning a better grade.

Learning Students self-reported (see below) that they learned more from this individual and group examination method in course exit surveys. Obviously they also self-reported that they like how it improves their grades. But just as important as self-reported results may be, in a first-semester second chance opportunity in Intermediate Accounting I course, 84.6% of the students who opted for a semester end gamble to improve their scores to a maximum of 75 on a previous poor score, scored higher than their first individual attempt. The gamblers’ average on the re-take was 14.2 points higher than the first attempt average. One third of the enrollment across three sections took the gamble.

As faculty, we want to provide as many learning opportunities as is feasible, and group examinations as a supplement to individual examinations are a feasible choice to help students succeed with a stronger foundation versus throwing in the towel when they encounter difficulty.

Decision and Planning Steps in the Individual and Group Examination Process

1. Make a decision about which courses you will try the individual and group quiz or examination technique, e.g., this professor has used this teaching technique in Intermediate Accounting I and II for all quizzes and examinations in the course, and in Auditing and Accounting Information Systems courses for weekly reading quizzes. While this effective learning strategy session shows the technique as used in Intermediate Accounting I and II course sequence, it can be applied to any accounting course.

2. Think about the time frame you can use for this type of testing as a learning technique. During the normal academic year, the students can take the individual portion of exams on one day and on a subsequent (or next) day they take the group exam. In this example, many of the second day’s exam questions are the same but with different numbers and additional new questions are added. The advantage to students of having another day or night to study before the group exam is that students have an incentive to retrace their areas of difficulty (see students comments below) and re-learn the material.

For quizzes, this professor has regularly done the individual and group portions back to back in the same class. This provides an active learning review of the material immediately after the first attempt. In the Intermediate Accounting sequence, this professor has done the individual and group examinations in one contiguous long period during the compressed summer schedule; this is an option if you give your exams in long periods outside of the regular class meeting period. The advantage this professor has observed is the vigorous debate which occurs during the development of the solutions and the immediate B.F. Skinner type feedback impact on learning, but the disadvantage is that it provides no immediate incentive to go back and re-study the material missed the first time. The other student disadvantage is the immediate emotional roller coaster ride students go through immediately when they realize how many items or points they missed; students do not have time to privately rein in their emotions.

3. Decide what percentage weights or scoring split you wish to give the individual and group exam components. The data in this example uses 75% for the individual and 25% for the group examination with the caveat that the weight added to each student’s exam is the average across all sections. Mathematically, if the average of the whole is not used as an equalizer or modifier, the lower performing students on the individual exam will receive many more points than the higher performing students.

4. Creating the actual exam is a decision process because the professor must consider what questions to keep the same or add; at a minimum the numerical amounts should be altered for a second day group exam. This process is not onerous if one of the publisher algorithmic homework problem generators is used to create exams for multiple sections. For quizzes in courses, consider using clickers to facilitate the group portion of a short reading quiz if time is a factor and the professor wants student to focus on the distinctions of the two most popular answers, e.g., in auditing or accounting information systems. In auditing, this professor has used the quiz and clicker technique to clarify standards.

5. Consider having all the students sign a group exam top sheet in which they commit to the use of the group score (an example follows below); this serves a reminder that this activity results in points and the students recognize that this not only matters to them but their peers.

6. Read through the “Process Steps in the Individual and Group Examinations” listed below. If you will be unable to grade the individual exam before the group exam and you have no quiz scores on this exam’s material to group students, consider using homework assignment scores as your first benchmark. On subsequent examinations you can use the first exam and subsequent quizzes, but this professor recommends mixing groups based upon the same exam’s individual scores because the material varies and students can be better or worse in subsequent material. The students should not know their group members until the group examination period, so studying outside of class engages everyone and not one particular study class exam group that is assigned all semester. In the example provided here, the students regularly remarked to the professor that they could not figure out how they were grouped but it always seemed to be different and it worked.

7. Before the quiz or examination in which you use this technique for the first time, spend a few minutes explaining it in class so students will be prepared to allocate additional study time after the individual exam for the group exam.

Process Steps in the Individual and Group Examinations

Step one: in an electronic spreadsheet, sort individual students by their first exam score, or by a recent quiz score;

Step two: distribute the students evenly by the aforementioned scores into groups, e.g. copy and paste back and forth or snaking via rows in an electronic spreadsheet (see example below); this should generate different groups for each exam;

Step three: after the group exam, in an electronic spreadsheet calculate your raw score based upon your predetermined percentages, e.g. 75 % of an individual score of 72 and 25% of a group score of 94 equals a new raw score of 77.5 (.75 * 72 + .25 * 94);

Step four: in an electronic spreadsheet calculate the differences between each student’s first original individual score and the new raw scores and determine the average of the differences;

Step five: if applicable, data across sections should be considered in discerning the group exam add on points for this exam or quiz;

Step six: make a decision on how many points to add for all students for this examination or quiz;

Step seven: announce or post the group results online.

What follows is an actual example of the steps described above.

STEP 1 Sort individuals by first exam score

STEP 4 Calculate difference for each student, then average

STEP 5 Look at data across sections, e.g., 5.26, 5.83 & 6.04

STEP 5 Look at data across sections, e.g., 5.26, 5.83 & 6.04

STEP 6 Determine points to add to all students

For fairness, the same points are

added to all students

The folloend of se

Q: Positi The extrathe mate

I also likundersta

Q: What

Individuastudents

exam stru

group ex

I really li

Stu

owing are soemester stude

ve character

a help and qrial.

ke the group and my mista

do you sugg

al and groupimprove the

ucture

ams

ike the group

udent Feedb

me typical eent survey:

ristics or stre

quizzes both

exams. It reakes. It reall

gest I keep th

p exams are ir grade.

p quizzes. 

back on Ind

excerpted res

engths of the

group and

eally helpedly reinforced

he same nex

a very nice f

ividual and

sponses to th

e course and/

individual d

d me reflect od the materi

t time?

feature of th

d Group Exa

he general op

/or instructo

definitely he

on what I diial.

is class. Gro

aminations

pen ended qu

r.

elped my und

id the day be

oup exam ob

uestions from

derstanding

efore and

bviously help

m an

of

ps

GROUPEXAM1

Name:____________________________________

Name:____________________________________

Name:____________________________________

Name:____________________________________

Name:____________________________________

Name:____________________________________

INITIAL IN BOX TO USE GROUP SCORE!

The syllabus for the course is on the web at

http://www.providence.edu/acc/cah/Intermediate%20Course%20Description%20Page.htm

Excerpts from the syllabus relating to the core competency learning objectives that the individual and group examinations described above are developing are the following:

Accountancy Major Learning Goals & Objectives (derived from AICPA Core Competency Framework to meet the Providence College Mission and

PC School of Business Mission)

Intermediate Accounting Course Specific Learning Goals & Objectives are in bold print below:

GOAL #2: COMPETENCE IN MAJOR Each student shall be able to demonstrate competence within his or her major field of study.

C. Functional Competencies Accountancy majors will develop functional competencies as they relate to the technical competence expected in the accounting profession in the following areas:

C. Functional Competencies - 1. Decision Modeling: Individuals preparing to enter the accounting profession must be able to use strategic and critical approaches to decision-making. They must objectively consider issues, identify alternatives, and choose and implement solution approaches in order to deliver services and provide value.

LEVEL 1: Identifies problems, potential solution approaches, and related uncertainties; LEVEL 2A: Organizes and evaluates information, alternatives, cost/benefits, risks and rewards of alternative scenarios; LEVEL 2B: Employs model-building techniques to quantify problems or test solutions; LEVEL 2C: Uses quantitative techniques to explore the likelihood of alternative scenarios;

C. Functional Competencies - 2. Leverage Technology: Technology is pervasive in the accounting profession. Individuals entering the accounting profession must acquire the necessary skills to use technology tools effectively and efficiently. These technology tools can be used both to develop and apply other functional competencies.

LEVEL 2A: Appropriately uses electronic spreadsheets and other software to build models and simulations;

C. Functional Competencies - 3. Measurement: Measures used should be both relevant (that is, bear on the decision to be made) and reliable (consistently measure what they purport to measure). Various measurement and disclosure criteria used by accounting professionals—such as US GAAP, IFRS, OCBOA (Other Comprehensive Basis of

Accounting) and tax reporting—have been codified to some degree. Other performance measures (such as Economic Value Added) or stated criteria (for example, investment performance) are used for special purposes. Some measurement criteria (such as effectiveness of internal control) are measured qualitatively, rather than quantitatively.

LEVEL 1A: Appropriately applies a given measurement method; LEVEL 1B: Identifies what needs to be measured; LEVEL 1C: Describes uncertainties about data and how items should be measured; LEVEL 2A: Describes the pros and cons of alternative methods of measurement; LEVEL 2B: Describes the implications of ambiguities when estimates are required;

C. Functional Competencies - 4. Reporting: Communicating the scope of work and findings or recommendations is an integral part of a professional service. An accounting professional in public practice might issue an audit or attestation report, recommendations for improved services, or tax or financial planning advice. An accounting professional in business, industry, or government might analyze operations or provide communications to the board of directors. Communicating clearly and objectively the work done and the resulting findings is critical to the value of the professional service. Some forms of communication are governed by professional standards (such as the form and content of the standard auditor's report or the required communications to the audit committees) or law. Others are based on the service applied and the needs of those to whom the accounting professional reports.

LEVEL 1: Lists types of information relevant to a given report;

D. Personal Competencies Accountancy majors will develop their personal competencies relating to the attitudes and behaviors of individuals preparing to enter the accounting profession. Developing the following personal competencies will enhance the way professional relationships are handled and facilitate individual learning and personal improvement.

D. Personal Competencies - 1. Communication: Accounting professionals are called upon to communicate financial and non-financial information so that it is understood by individuals with diverse capabilities and interests. Individuals entering the accounting profession should have the skills necessary to give and exchange information within a meaningful context and with appropriate delivery. They should have the ability to listen, deliver powerful presentations and produce examples of effective business writing.

LEVEL 1A: Identifies uncertainties about the best way to communicate; LEVEL 1B: Expresses information and concepts with conciseness and clarity when writing and speaking;

D. Personal Competencies - 2. Interaction: Accounting professionals must be able to work with others to accomplish objectives. This requires them to act as valuable business partners within organizations and markets and work in teams to provide business solutions. Thus, individuals entering the accounting profession should demonstrate an

ability to work productively with individuals in a diversity of roles and with varying interests in the outcome.

D. Personal Competencies - 5. Problem Solving / Decision Making: Accounting professionals are often asked to discern the true nature of a situation and then determine the principles and techniques needed to solve problems or make judgments. Thus, individuals entering the accounting profession should display effective problem solving and decision-making skills, good insight and judgment, as well as innovative and creative thinking.

LEVEL 1A: Lists information and evidence that is relevant for a problem; LEVEL 1B: Identifies uncertainties about the interpretation or significance of information and evidence;

Course Assessment: Student achievement of the courses' learning objectives will be assessed by the following:

o [Quizzes and examinations to measure achievement, proficiency and mastery of the course's learning objectives, specifically the functional competency of measurement.]