updated september 2016 - theiia documents/item-writers-guide.pdf · most items for the iia’s...

38

Upload: others

Post on 08-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who
cbrune
Typewritten Text
Updated September 2016
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
cbrune
Typewritten Text
Page 2: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 1 — Item Writer’s Guide

Welcome Message

Welcome to The IIA’s Item Writer’s Guide!

Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come

from writers like you, who have senior-level experience, hold an IIA certification, and

enjoy contributing to the growing global profession of internal auditing. Our team of

dedicated item writers consistently describes the IIA Certification Item Writing Program

as both rewarding and challenging.

This guide was created to help you succeed as an IIA exam item writer. Within these

pages, you’ll find guidance on the item writing process, how to write a quality item,

common mistakes to avoid, and an array of examples and additional resources to

support you.

Item writing is both an art and a skill, and your item writing skills are certain to improve

over time. It takes practice and patience, so don’t be discouraged if your items are not

accepted initially. Aim for quality over quantity.

Additionally, to develop your item writing skills, you are invited to join our IIA

Certification Item Writer’s LinkedIn Group and take advantage of the writer support

services listed in Chapter 6. If you did not receive an email invitation to join the Item

Writer’s LinkedIn Group, or if you have any questions or require additional information

about the item writing process, please email us at [email protected].

Good luck!

*Note: This guide is confidential, and its use is permitted only by those who have been accepted into The

IIA Certification Item Writing Program.

Page 3: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 2 — Item Writer’s Guide

Table of Contents

Chapter 1: Item Submission Process p4

Process Overview

Review and Approval

Writer Notification

Payment and Rewards

Chapter 2: What You Should Know Before You Begin p7

Multiple-choice Format

Anatomy of an Item

Approved References

Item Difficulty: Awareness vs. Proficiency

Chapter 3: How to Write an Item p12

10 Steps for Writing an Item

Item Review Checklist

Chapter 4: 10 Tips for Success p14

Write on a Single Exam Syllabus Topic

Use Approved References

Focus on a Key Principle or Concept, Not a Trivial Detail

Write for a Global Audience

Write a Clear, Concise Stem

Avoid Hints

Write One Clearly Correct Option

Write Relevant, Plausible Distractors

Use Positive Language

Write a Well-supported Rationale

Page 4: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 3 — Item Writer’s Guide

Chapter 5: Common Reasons Items Are Declined p25

Chapter 6: Writer Support Services p27

Social Networking

Collaborative Writing

Item Writing Workshops

Virtual Item Writing Workshops

Online Tutorials

Helpful Links

Appendix: Item Examples p29

Awareness Level

Proficiency Level

Previous Examples of Successful Items

Page 5: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 4 — Item Writer’s Guide

Chapter 1: Item Submission Process

Process Overview

Now that you have been accepted into the IIA Certification Item Writing Program, you

will be contacted each month by email and invited to write items for one of the IIA

certifications that you currently hold. The monthly call for items will specify the

certification for which we are seeking items and include a list of approximately 10-12

topic areas that correspond to specific knowledge areas from the Exam Syllabus.

We understand that some months you may be unable to submit any items. However,

we expect writers to submit a minimum of 20 items per year, per the IIA Certification

Item Writing Program agreement.

To submit an item, you must complete the Microsoft Excel Item Writing Template

provided each month. If you do not have access to Microsoft Excel or are having a

problem with the template, please contact the IIA Certification Department at

[email protected].

Review and Approval

Once we receive your items, they go through a rigorous review by IIA headquarters

Certifications staff to ensure all areas of the template are complete. Please note that

incomplete items will be returned to the writer automatically. The Certifications staff

will check your references and make minor corrections to your item to improve

grammar and correct spelling. However, if there are too many problems with the item, it

will be declined. Items that pass the Certifications staff review will be sent to The IIA’s

global Exam Development Committee (EDC) for final review and approval.

Writer Notification

You will be advised by email of the number of items accepted and/or declined from your

monthly submission as well as any additional information or guidance provided by the

reviewers to assist your learning process. Typically, you will be notified within 60 days of

your submission. However, there may be high-volume periods when additional time is

needed to process, review, and approve items.

If your items are declined, we attempt to provide feedback to help you understand the

reasoning as well as tips and guidance on how to strengthen your current item or create

a new one. However, please understand that feedback on individual items is sometimes

Page 6: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 5 — Item Writer’s Guide

not possible due the volume of items being processed. (See Chapter 5 for a list of

common reasons items are declined.)

Payment and Rewards

The IIA has a robust rewards program for successful individual writers and groups of

writers, known as Test Development Teams (TDTs). Both are eligible to earn continuing

professional education (CPE) credits for accepted items. Individual writers also can

choose to receive monetary payment, an IIA Research Foundation Bookstore gift

certificate, or a combination of the two. The host chapter of the TDT is eligible to

receive Chapter Achievement Program (CAP) points for accepted items.

FOR INDIVIDUAL WRITERS

1. 1 CPE For each item accepted

CIA writers can receive a maximum of 10

CPE credits per year.

Writers for our specialty exams can

receive a maximum of 5 CPE credits per

year.

2. (Choose

payment,

gift

certificate,

or

combination)

$50 For each item accepted

1. Check or Wire Transfer Payment Payment will be made in the form of a

check or wire transfer. If a wire

transfer is requested, The IIA will pay

the delivery fee only (up to US $30).

The writer will be responsible for any

additional fees.

2. IIA Research Foundation Bookstore

Gift Certificate The gift certificate will be delivered to

the writer electronically and can be

used toward online purchases:

http://www.theiia.org/bookstore.

**The writer can elect to have the payment

amount split between options 1 and 2.

Page 7: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 6 — Item Writer’s Guide

FOR TEST DEVELOPMENT TEAMS (TDTs)

1. 1 CPE For each item accepted

CIA writers can receive a maximum of 10

CPE credits per year.

Writers for our specialty exams can receive

a maximum of 5 CPE credits per year.

2. 1 CAP

Point For each item accepted

Chapter Achievement Program (CAP)

points are awarded to the host chapter of

the Test Development Team.

If you have questions regarding payment or rewards, please contact the IIA Certification

Department at [email protected].

Page 8: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 7 — Item Writer’s Guide

Chapter 2: What You Should Know Before You Begin

Multiple-choice Format

The IIA’s certification exams feature multiple-choice items that include a question and

four possible responses. Previously, we accepted items in two formats (depicted below):

1. Single Response.

2. Numeric Combined Response.

However, we are no longer accepting Numeric Combined Response items.

Figure 1: Single Response

Figure 2: Numeric Combined Response

According to IIA guidance, independence of the internal audit activity is achieved

through which of the following?

1. Staffing and supervision.

2. Human relations and communications.

3. Functional reporting to the board.

4. Unrestricted access to senior management.

A. 1 only.

B. 1 and 3.

C. 3 and 4 only.

D. 2, 3 and 4.

According to IIA guidance, independence of the internal audit activity is achieved

through which of the following?

A. Staffing and supervision.

B. Continuing development and due professional care.

C. Human relations and communications.

D. Organizational status and objectivity.

Page 9: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 8 — Item Writer’s Guide

Anatomy of an Item

Each item will contain all of the following components:

Topic The subject area being tested, which corresponds to a specific topic

area from the appropriate Exam Syllabus.

Stem The initial part of the item, which ends with a question. The stem length

may vary considerably, though most are one or two sentences (10 to 35

words).

Options The four possible responses, consisting of one key and three distractors.

Key The correct option.

Distractor The incorrect options.

Rationale An explanation of why each option is correct or incorrect.

Reference The publication title(s) and page number(s) that support the rationale.

Note that the Microsoft Excel Item Writing Template you will use to submit items each

month includes a cell for each of these components. The Topic and Reference cells also

feature dropdown menus with prepopulated options from which to select.

Figure 3: Sample Item in the Template

Page 10: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 9 — Item Writer’s Guide

Approved References

As mentioned above, each item must contain a reference or multiple references that

support the rationale and “prove” the correctness of the key. A list of approved

references is available for each IIA exam. Item writers must use the references from this

list to support their item.

Additionally, a detailed CIA Reference Map is available to guide writers to specific

reference titles and chapters that pertain to each topic on the exam syllabus. Item

writers who write for the CIA exam will receive a current Reference Map each month

with the call-for-items email. CRMA and CGAP Reference Maps are in development and

will be available at a later date.

Item Difficulty: Awareness vs. Proficiency

For each IIA certification exam, there is a detailed syllabus that outlines the specific

topic areas covered on the exam. Each topic area on the exam syllabus is designated as

either an Awareness Level (A) or Proficiency Level (P) item.

Awareness Level Items — Awareness Level items are designed to test a candidate’s

basic understanding of the topic, including terms, fundamental concepts, and principles.

They should avoid simple glossary definitions of a term or material copied straight from

the IPPF or reference books. They should not require a candidate to have memorized a

list. Instead, a well-crafted Awareness Level item tests a candidate’s ability to identify a

concept or principle in the context of a given example.

The following are two examples of an Awareness Level item about of the IIA Code of

Ethics. The first example is unsuccessful because it merely tests candidates’

memorization of the Code of Ethics components, not their understanding of the

principles. The second example is better, as it requires a candidate to understand and

identify what competency actually means in context of the scenario.

Page 11: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 10 — Item Writer’s Guide

Figure 4: Requires Out-of-context Memorization

Figure 5: Appropriate Awareness Level Item

Proficiency Level Items — Proficiency Level items require candidates to demonstrate a

thorough understanding of the topic and then to apply this understanding to real-world

situations, drawing on their knowledge, experience, reasoning skills, judgment, and

common sense. A great Proficiency Level item has a stem that’s challenging, thought

provoking, and relevant to today’s practice of internal auditing. It may describe a

fictitious scenario and test the candidate’s ability to solve a problem or respond

appropriately to the situation, applying higher-level concepts and good judgment.

Below is an example of a Proficiency Level item for the CIA Part 2 exam [syllabus topic

“(II.D.3) Conduct follow-up and report on management's response to internal audit

recommendations (P)”].

Which of the following are the four principles that comprise the essential

elements of the IIA Code of Ethics?

A. Integrity, Trust, Objectivity, and Accountability.

B. Integrity, Objectivity, Trust, and Competency.

C. Integrity, Objectivity, Confidentiality, and Competency.

D. Integrity, Objectivity, Confidentiality, and Accountability.

A new auditor determines that the audit program will require his use of advanced

statistical methods. Given that he has not been trained in these methods, he opts

to forward this work to a more experienced senior auditor. This decision

demonstrates which of the following IIA Code of Ethics attributes?

A. Objectivity.

B. Integrity.

C. Competency.

D. Confidentiality.

Page 12: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 11 — Item Writer’s Guide

Figure 6: Successful Proficiency Level Item

Whether you’re writing an item at the Awareness or Proficiency level of difficulty, it’s

important to remember that the average IIA certification candidate has four years of

college-level education and two years of professional experience. Additionally, IIA

exams are offered in 18 languages in more than 100 countries around the world.

Therefore, when writing an item, you should consider:

1. The extent to which a candidate is likely to have experienced (or should have

studied) the topic being tested.

2. How well a candidate needs to know the topic to answer the item correctly.

Use good judgment to combine these two factors and create items at the specified level

of difficulty. Your item should not be obvious or easy to answer if the student is poorly

prepared. A poorly prepared candidate may have memorized basic terminology but will

be challenged to apply that level of understanding to the more complex and diverse

Proficiency Level items on the exam. Likewise, your item must not require such a high

level of technical knowledge and expertise that the average candidate would not have

the specialized knowledge and experience to answer it correctly.

See the Appendix for additional examples of Awareness and Proficiency Level items.

Page 13: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 12 — Item Writer’s Guide

Chapter 3: How to Write an Item

10 Steps for Writing an Item

1. SELECT TOPIC – When you receive the monthly call-for-items email, note the specific

exam and topic areas for which items are being requested. The topic you select will

determine whether you will be writing an Awareness or Proficiency question.

2. CONSULT REFERENCE MAP – Use the appropriate Reference Map to identify the

approved references and relevant pages that pertain to the topic you selected.

3. REVIEW REFERENCES – Read the reference passages you selected. Ensure you have a

good understanding of the topic and how the reference pertains to it.

4. FORMULATE IDEA – Next, reflect on real-life experiences to formulate an idea for

your item.

5. WRITE STEM & KEY – Write a concise stem and correct response (key) that would

effectively test candidates on the single topic area you selected.

6. WRITE DISTRACTORS – Draw upon personal experiences to create strong distractors

that would appear reasonable to a candidate who does not have a complete

command of the concept being tested. A well-written distractor may incorporate

common mistakes, inappropriate actions, or misconceptions regarding the topic.

7. WRITE RATIONALE – Explain why each option is correct or incorrect, citing specific

reference passages and page numbers when applicable. Write a separate and

complete rationale for each of the four options.

8. PAUSE – Walk away from your item. When you reread your item at a later time, you

likely will have gained a fresh perspective, perhaps new ideas. You will be able to

look at your item objectively and identify opportunities for improvement.

9. CHECK SPELLING & GRAMMAR – Some writers compose their items in Microsoft

Word and take advantage of the software’s spelling and grammar check prior to

transferring (copying and pasting) their items to the item writing template.

10. REVIEW CHECKLIST – Use the following Item Review Checklist to check for

completeness and quality prior to submitting your item.

Page 14: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 13 — Item Writer’s Guide

Item Review Checklist

1. All relevant sections of the item writing template are complete.

2. The item addresses the topic selected from the exam syllabus.

3. The item assesses a key principle or concept, not a trivial fact.

4. The item tests a candidate’s knowledge on a topic and does not trick the candidate.

5. The item is appropriately Awareness Level or Proficiency Level.

6. The Awareness Level item avoids simple definitions or material copied straight from

reference books.

7. The Proficiency Level item requires the prepared candidate to apply judgment,

experience, and knowledge to answer the item correctly.

8. The difficulty level is appropriate for a candidate with a college-level education and

two years of professional experience.

9. The item can be translated and applied globally (avoids slang, idioms, and regional or

culturally-specific information).

10. The stem is clear and concise with no superfluous information.

11. The stem is phrased positively, as much as possible (avoids terms such as least, not,

false).

12. The item is free from hints such as exclusive terms (all, only, never, always) or

repetitive terms that appear in both the stem and the key.

13. All options are similar in length and format.

14. All options are aligned with the topic contained in the stem.

15. There is only one correct option (the key).

16. All distractors are plausible, but definitely incorrect.

17. The item does not contain any weak distractors that a candidate could dismiss easily.

18. The rationale articulates separately and clearly why the key is correct and why each

distractor is incorrect, and specific reference passages are cited when applicable.

19. The cited reference(s) support the rationale and choice of the key response.

20. Grammar, punctuation, and spelling are accurate.

Page 15: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 14 — Item Writer’s Guide

Chapter 4: 10 Tips for Success

1. Write on a single exam syllabus topic. The following item was written to test candidates on a specific topic from the CIA

Part 2 exam syllabus: “(I.A.6) Communicate internal audit key performance

indicators to senior management and the board on a regular basis (P).”

Figure 7: Focused on Exam Syllabus Topic

By contrast, the next item fails to align with the exam syllabus topic selected by the

writer: CIA Part 1 Exam, “(II.B) Management control techniques (A).”

Figure 8: Fails to Align With Exam Syllabus Topic

Page 16: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 15 — Item Writer’s Guide

In all other respects, the previous item is a quality item. However, it would be more appropriate for the CIA Part 1 Exam, “(III.B) Data Analysis & Interpretation (P).”

2. Use approved references.

Use the appropriate Reference Map to identify the approved references for each

topic on the exam syllabus. For example, as shown below, when writing on the

CIA Part 1 Exam topic “(I.C.3) Determine if the required knowledge, skills, and

competencies are available (P)” writers may use any of the three references

listed on the CIA Part One Reference Map: the IPPF, Internal Auditing: Assurance

and Consulting Services, or Sawyer’s Guide for Internal Auditors.

Domain Description IPPF Practice Guides

Textbook (Internal Auditing: Assurance and Advisory Services)

Sawyer’s Guide for Internal Auditors

I.C.3 Determine if the required knowledge, skills, and competencies are available (P)

IPPF (Stds & PAs)

Chapter 9: Managing the Internal Audit Function

V1-Chapter 4: Audit Process

Figure 9: Excerpt From the CIA Part One Reference Map

Be careful when drawing conclusions or making assumptions based on the

reference material. If a textbook states, “Skimming is a common type of fraud,”

don’t interpret that to mean, “Skimming is the most common type of fraud.”

Likewise, don’t assume that lists are all-inclusive. For example, if a reference

book lists skills that auditors should possess, don’t assume that because a

particular skill isn’t included on that list, it must not be a valuable skill for an

auditor to possess.

3. Focus on a key principle or concept, not a trivial detail. In the first example below, the item focuses on a trivial fact about Standard 1100.

As currently written, the item would only serve to test candidates on the names

of individual standards, rather than the concepts behind them.

Page 17: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 16 — Item Writer’s Guide

Figure 10: Focuses on a Trivial Detail

The following Awareness Level item is better, as it tests the candidate on the

broader concept of maintaining organizational independence. (It would be better

still if all options were similar in length.)

Figure 11: Focuses on Key Concept

The IPPF includes standards on which of the following?

A. Independence and Objectivity.

B. Competency.

C. Confidentiality.

D. Integrity.

Rationale A – Correct. Standard 1000 is named “Independence and Objectivity.”

B – Incorrect. Competency is not a standard, but part of the Code of Ethics.

C – Incorrect. Confidentiality is not a standard, but part of the Code of Ethics.

D – Incorrect. Integrity is not a standard, but part of the Code of Ethics.

Which of the following statements describes the primary reason the chief

audit executive should report to the board?

A. The board has the final authority to approve expenses.

B. The board’s organizationwide perspective makes it well positioned to

determine where internal audit activity resources should be deployed.

C. The board is well positioned to protect the auditor’s freedom to carry out

her responsibilities in an unbiased manner.

D. The board audit committee typically has the most knowledge about the

organization’s risks.

Rationale A – Incorrect. Approval of expenses is not the primary reason for this reporting relationship.

B – Incorrect. The CAE is responsible for determining where internal audit activity (IAA)

resources should be deployed (per Standard 2030- Resource Management).

C – Correct. The CAE must report to a high enough level in the organization to allow the IAA to

fulfill its responsibilities (per Standards 1100 and 1110).

D – Incorrect. The AC typically doesn’t have the most knowledge about the organization’s risks.

Page 18: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 17 — Item Writer’s Guide

4. Write for a global audience. Keep in mind that your successful items will be translated into 18 languages.

Avoid idioms, slang terms, and culturally specific information. Likewise, avoid

items that are specific to a country or region, as in the following example.

Figure 12: U.S.-specific

5. Write a clear, concise stem. Stems should include only the information needed to arrive at the correct

response. Avoid long sentences, complex scenarios, and irrelevant details. Again,

the goal is not to trick exam candidates. The subject being tested should be

readily apparent. Additionally, the stem should avoid “teaching,” or providing

information that the candidate should already know.

The following example includes details that aren’t necessary for this particular

item, as its purpose is to test a candidate’s understanding of how reporting to the

Which of the following risk management frameworks is extensively used

within U.S. federal agencies?

A. NIST SP 800-37.

B. CoCo.

C. Turnbull.

D. King II.

Rationale A – Correct. The National Institute of Standards and Technology in the United States

developed this framework, which is used extensively in U.S. federal agencies. (Sawyer’s 6th

edition, Vol. 1, p 56)

B – Incorrect. CoCo is the Canadian Institute of Chartered Accountants’ “Criteria of Control”

report. (Sawyer’s 6th edition, Vol. 1, p 40)

C – Incorrect. The Turnbull Report covers best practice on internal control for UK listed

companies.

D – Incorrect. King II is a corporate governance report released by the Institute of Directors in

Southern Africa.

Page 19: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 18 — Item Writer’s Guide

board protects the internal audit activity’s independence and objectivity. (Figure

11 above shows how the stem for this item can be simplified.)

Figure 13: Unnecessary Details in Stem

The next two examples further illustrate the concept of teaching and how a stem

can be rewritten for conciseness and to avoid teaching.

Figure 14: Teaching in the Stem

Figure 15: Concise Stem

6. Avoid hints. The stem and options should be free from signals such as grammar, a noticeable

difference in style and language, or repetitive terminology that could

inadvertently reveal the key. Avoid using absolutes (such as always, never, only,

or all) in distractors, as test-wise candidates know statements with absolutes are

seldom true. (See Figure 23, option D.)

A chief audit executive (CAE) was recently hired at a chemical company, and

the board has asked her to develop a new internal audit activity (IAA) in the

organization. She gets to work creating a charter for the new IAA. The

charter outlines reporting responsibilities for the IAA and includes a clause

indicating that the CAE will report to the board. Which of the following

statements justifies the inclusion of this clause?

According to IIA guidance, the chief audit executive must ensure that the

purpose, authority, and responsibility of the internal audit activity is reflected

in the audit charter. Which of the following would be considered an

appropriate example of this responsibility?

According to IIA guidance, which of the following is a responsibility of the

internal audit activity that should be included in the audit charter?

Page 20: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 19 — Item Writer’s Guide

In the following example, using the term groups in the stem inadvertently reveals

the key (makes the item “keyable”), as the board is the only option that qualifies

as a group.

Figure 16: Cueing Reveals the Key

One way to avoid hints is to ensure that options are parallel in grammar,

sentence structure, and length. In the following example, option C is longer and

structurally different than the distractors, making the key obvious.

Figure 17: Key Appears Different Than Distractors

7. Write one clearly correct option. Each item should include one clearly correct option (the key) and three clearly

incorrect options (distractors). Exercise care when writing stems that include

terms such as best, most, least, or typically. These terms allow you to create

challenging Proficiency Level items, but they can also introduce subjectivity. If

Which of the following groups is responsible for approving the internal audit

charter?

A. The board.

B. The CEO.

C. The chief financial officer.

D. The chief audit executive.

According to IIA guidance, which of the following describes the primary

reason the chief audit executive should report to the board?

A. The board can approve expenses.

B. The board can determine where to deploy internal audit resources.

C. When the internal audit activity reports to a high level in the organization,

audit independence is protected.

D. The board is most knowledgeable about risk.

Page 21: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 20 — Item Writer’s Guide

you use one of these terms, your rationale must explain why the distractors are

clearly wrong or inferior to the correct answer.

Avoid creating distractors that are incorrect simply because of a subtle

technicality or hair-splitting. Remember, the goal of item writing is not to trick

exam candidates, but to test their knowledge of the subject material.

In the following example, options A and C are clearly incorrect. Option B is the

key. However, option D is also correct. The author’s attempt to explain why

option D is incorrect (in the rationale) is merely an unsubstantiated personal

judgment.

Figure 18: Lacks One Clearly Correct Option

Although the example below uses the term most, it is successful because the

rationale is able to justify the key by explaining why options A and C are

unquestionably less efficient and effective than option D.

Which of the following is the most important reason for the chief audit

executive to ensure that the internal audit activity is well resourced?

A. To ensure there is some protection from the impacts of outsourcing.

B. To demonstrate sufficient capability to meet the audit plan requirements.

C. To establish credibility with the audit committee and management.

D. To ensure that the internal audit activity adds value to the organization.

Rationale A – Incorrect. Seeking to avoid the negative impacts from outsourcing should not be the CAE’s

primary concern and therefore is not the most important reason to ensure that the internal

audit activity is well resourced.

B – Correct. Standard 2030 requires that resources are sufficient to achieve the approved plan.

C – Incorrect. Credibility is not established through resourcing, but rather the performance of

the internal audit activity overall.

D – Incorrect. Standard 2000 states that the CAE must effectively manage the internal audit

activity to ensure that it adds value to the organization. However, this is a broad goal, so I

don’t think it is as important as the more specific Standard 2030, which details the CAE’s

responsibilities to ensure adequate resources.

Page 22: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 21 — Item Writer’s Guide

Figure 19: Contains One Clearly Correct Option

8. Write relevant, plausible distractors. It’s important to create distractors that a reasonable person — but someone who

isn’t knowledgeable about the topic — might select. This may be the most

difficult aspect of item writing, yet it is critical. Even one weak distractor is

sufficient to bias the item. Partially correct information, common misinformation,

and logical misinterpretations make good distractors. It may help to start with

two or three correct answers and consider how you can alter them to make them

incorrect.

Avoid writing distractors that group the options into two categories, which would

allow the candidate to dismiss quickly two of the options. In the following item,

for example, a candidate who knows the answer is yes can immediately dismiss

options C and D.

Figure 20: Distractors Fall Into Two Categories

Options A. Yes, because the board approved the charter. B. Yes, because management approved the charter. C. No, because the board does not need to approve the charter. D. No, because management does not need to approve the charter.

Page 23: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 22 — Item Writer’s Guide

Below are two examples of well written, plausible distractors.

Figure 21: Plausible Distractors

Figure 22: Well-balanced Distractors

Page 24: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 23 — Item Writer’s Guide

9. Use positive language. Stems should include positive terms (e.g., true, effective, and appropriate) as

opposed to negative terms (e.g., not, false, and least). Positive items are more

difficult to write, as they require the writer to create three untrue, but plausible

options. The following is an example of a negative item. Additionally, this item

contains cueing. The false statement includes the word only, as test-wise

examinees would suspect.

Figure 23: Negative Stem

10. Write a well-supported rationale.

The rationale is critical to justify the validity of your item and the option you

selected as the key. It should include a unique explanation for each option and

even distinct reference titles, quotes, and page numbers when applicable. If your

item requires mathematical calculations, show how the solution is calculated.

Think of the rationale as your opportunity to explain why your distractors are

plausible and your key is undoubtedly the only correct option. Explaining why an

option is wrong is as important as explaining why an option is correct. For

Proficiency Level items, avoid simply quoting the reference materials. If you

Which of the following statements is false regarding the IIA Code of

Conduct?

A. It requires internal auditors to uphold the principles of integrity and

objectivity.

B. It includes rules of conduct regarding internal audit competency.

C. It aims to promote an ethical culture in the profession of internal auditing.

D. It applies only to individuals who perform internal audit services.

Rationale A – Incorrect. This is true. The Code of Ethics includes the following principles: integrity,

objectivity, confidentiality, and competency.

B – Incorrect. This is true. The Code of Conduct includes rules of conduct regarding integrity,

objectivity, confidentiality, and competency (IPPF 2013, pp6-7).

C – Incorrect. This is true (IPPF 2013, p4).

D – Correct. This is false. The Code of Ethics applies to both entities and individuals that

perform internal audit services (IPPF 2013, p5).

Page 25: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 24 — Item Writer’s Guide

include a quote, also explain how the passage can be applied to your particular

item and how it justifies the correctness or incorrectness of your option(s).

Writing the rationale is the step where you are most likely to discover problems

with your item such as multiple correct options, an item that doesn’t align with

the selected topic area, or references that don’t fully support the rationale.

In the first example below, the author failed to explain why options A, B, and C

are incorrect. In the second example, the rationale demonstrates how each

option is directly related to the topic of objectivity and clearly explains why each

option is either correct or incorrect.

Figure 24: Inadequate Rationale

Figure 25: Robust Rationale

Rationale A, B, C – Incorrect. These are all true statements.

D – Correct. This statement is false. The Code of Ethics applies to both entities and

individuals that perform internal audit services (IPPF 2013, p5).

Page 26: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 25 — Item Writer’s Guide

Chapter 5: Common Reasons Items Are Declined

Not Relevant The concept is not one that we wish to test – either because it

does not appear on the exam syllabus, or because it is not a

knowledge area that the candidate could be reasonably expected

to know in the context of this certification. (See Chapter 4, Tip #

1 for guidance on writing items that align with a single exam

syllabus topic.)

Keyable The item contains grammar problems, repetitive terminology, or

other hints that inadvertently reveal the key. (See Chapter 4, Tip

#6 for guidance on how to avoid hints.)

Not Global The item contains country-specific standards or legislation,

idioms, slang terms, or other culturally specific information not

applicable to a global audience. (See Chapter 4, Tip #4 for

guidance on writing for a global audience.)

Confusing The item contains insufficient or too much information, has

grammar or vocabulary issues, tests more than one topic, or is

otherwise unclear. (See Chapter 4, Tip #5 for guidance on writing

a clear, concise stem.)

Too Difficult Although the item topic is aligned with the exam syllabus, the

item includes concepts that are not common practice or are too

advanced for the average candidate. (Click here for guidance on

achieving the desired level of difficulty.)

Too Basic The item is not sufficiently challenging for the topic area. Items

that fall into this category include those that are too definitional,

require mere memorization, test a trivial detail, or otherwise fail

to achieve the desired Awareness or Proficiency level. Often, the

correct answer is obvious. (See Chapter 4, Tip #3 for guidance on

assessing key principles and review Chapter 2 more information

on Awareness Level versus Proficiency Level items.)

Too Subjective The item is biased, vague, or requires an unreasonable amount of

judgment from the candidate to select the correct answer. The

item may include more than one correct answer. (See Chapter 4,

Page 27: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 26 — Item Writer’s Guide

Tip #7 for guidance on writing one clearly correct option and

three clearly incorrect distractors.)

Weak Distractors One or more distractors lack quality, plausibility, or

reasonableness and can be dismissed easily. (See Chapter 4, Tip

#8 for guidance on how to write relevant, plausible distractors.)

Inadequate Reference The reference title or page numbers cited do not support the

item, or the number of pages cited is too large (e.g., pp 20-28).

(See Chapter 4, Tip #2 for guidance on avoiding the common

error of making assumptions or drawing conclusions

inappropriately from the reference material.)

Inadequate Rationale The rationale fails to justify the validity of the item, does not

include separate concrete explanations for why each option is

either correct or incorrect, or is not supported by the reference

cited. (See Chapter 4, Tip #10 for guidance on writing a well-

supported rationale.)

Page 28: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 27 — Item Writer’s Guide

Chapter 6: Writer Support Services

Social Networking

As a participant in The IIA’s Exam Writing Program, you are invited to join our exclusive

Exam Writers Group on LinkedIn. This invitation-only group offers a private forum for

connecting with IIA item reviewers and other item writers, enabling you to ask

questions, share writing tips, and seek advice. When you were accepted into the Item

Writing Program, you should have received an invitation with a link to this “hidden”

group. If you need assistance or have any questions, please contact the IIA Certification

Department at [email protected].

Collaborative Writing

Test Development Teams (TDTs) consist of auditors who meet to develop items in a

collaborative environment led by a trained facilitator. TDTs offer an opportunity for peer

review of items as they are being developed. The approach allows writers to network

while contributing to the profession and producing high quality items. If you’re

interested in developing a TDT in your area, please contact the IIA Certification

Department at [email protected].

Item Writing Workshops

The IIA occasionally offers Item Writing Workshops, allowing writers in a geographical

area to collaborate by participating in a one- or two-day workshop led by an IIA item

reviewer. Item writers will be notified of workshop opportunities available in their area.

Online Peer Review Workshops

The IIA offers quarterly item writing and peer review workshops in a virtual setting,

allowing item writers from a variety of geographical areas to participate, learn from the

item reviewer, ask questions, and share best practices. Item writers will be notified via

email of upcoming virtual workshop opportunities. These workshops are not mandatory

but are highly recommended, especially for new writers.

Page 29: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 28 — Item Writer’s Guide

Video Tutorials

Prerecorded training videos are available online to help item writers fine-tune their

writing skills by learning best practices and identifying common pitfalls to avoid. The

following tutorials can be found by visiting the Item Writers Only Webpage:

Item Writing – The Basics

How to use the Item Writers Microsoft Excel Template

10 Tips for Better Item Writing

Helpful Links

Item Writers Only Webpage

Glossary of internal audit-related terms

Approved CIA references

Approved CRMA references

Information on redeeming points earned for reference materials

Video tutorials

CIA Exam Syllabus

CRMA Exam Syllabus

CCSA Exam Syllabus

CGAP Exam Syllabus

CFSA Exam Syllabus

Certifications Web Pages

Start a Test Development Team (TDT)

*Additional reference maps for the CGAP, CFSA, and CCSA exams will be available soon.

Page 30: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 29 — Item Writer’s Guide

Appendix: Item Examples

On the following pages, you’ll find six Awareness Level examples, 10 Proficiency Level

examples, and a listing of the successful items included earlier in this guide.

Awareness Level Examples

Figure A.1: Awareness Level

Figure A.2: Awareness Level

Page 31: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 30 — Item Writer’s Guide

Figure A.3: Awareness Level

Figure A.4: Awareness Level

Page 32: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 31 — Item Writer’s Guide

Figure A.5: Awareness Level

Figure A.6: Awareness Level

Page 33: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 32 — Item Writer’s Guide

Proficiency Level Examples

Figure A.7: Proficiency Level

Figure A.8: Proficiency Level

Page 34: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 33 — Item Writer’s Guide

Figure A.9: Proficiency Level

Figure A.10: Proficiency Level

Page 35: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 34 — Item Writer’s Guide

Figure A.11: Proficiency Level

Figure A.12: Proficiency Level

Page 36: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 35 — Item Writer’s Guide

Figure A.13: Proficiency Level

Figure A.14: Proficiency Level

Page 37: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 36 — Item Writer’s Guide

Figure A.15: Proficiency Level

Figure A.16: Proficiency Level

Page 38: Updated September 2016 - TheIIA Documents/Item-Writers-Guide.pdf · Most items for The IIA’s certification exams (CIA, CCSA, CGAP, CFSA, and CRMA) come from writers like you, who

IIA Certifications — 37 — Item Writer’s Guide

Previous Examples of Successful Items

Figure 1: Single Response

Figure 5: Appropriate Awareness Level Item

Figure 6: Successful Proficiency Level Item

Figure 7: Focused on Exam Syllabus Topic

Figure 11: Focuses on Key Concept

Figure 19: Contains One Clearly Correct Option

Figure 21: Plausible Distractors

Figure 22: Well-balanced Distractors

Figure 25: Robust Rationale