annual exam report 2013

8/13/2019 Annual Exam Report 2013

1/25


2/25

Published by Deakin University on behalf of CPA Australia Ltd, ABN 64 008 392 452

CPA Australia Ltd 2014 (Edition 14a)

The contents and any information contained in this document (Information) are for general information only. They are not

intended as professional advice. For any professional advice, please consult a suitable qualified professional. CPA AustraliaLtd, Deakin University and the author(s) of the Information (Entities) make no representation about the content and suitability

of this Information for any purpose. The Entities disclaim all warranties with regard to the contents and in no event will be

liable for any loss and/or damage whatsoever resulting from loss of income or profits, whether in an action of contract,

negligence or other tortious action, arising in connection with the use and performance of and/or reliance of the Information.


3/25

Leading the world in accounting educationThe CPA Program is a leading professional education program in accounting, finance and business.

Its success is reflected in its continual growth for over 25 years. The Program is portable, international and

requires self-directed and independent learning. It is a complete education and practical experience program,

carefully structured to provide maximum benefit to candidates now and as they progress in their careers.

CPA Australia undertakes continuous improvement to enhance the CPA Programs global relevance andcurrency of content, and provide more ways for people to develop a career built on professional accounting

skills. CPA Australia provides entry pathways designed to offer flexible entry modes for those who have not

previously completed an accounting degree. These changes are part of CPA Australia realising our goal to

maximise the share of people who want a career built on professional accounting skills.

Recognising the complex business environment in which organisations operate, CPA Australia has a broad

education program that is designed to produce high-quality graduates with well-rounded business skills

and sound technical competencies.

The CPA Program consists of two levels:

Foundation levelcore knowledge requirements that can be fulfilled through CPA Australias

eight foundation level exams or through an accredited or recognised higher-education degree program.

Professional levelsix postgraduate level segments plus three years of mentored experience in an

accounting, finance or business role.

The syllabus for the CPA Program focuses on strategy, leadership and international business. The content

of the Program is globally relevant, with a focus on providing flexibility of learning and delivery modes.

The CPA Program is designed to meet the requirements of the International Education Standards issued by

the International Federation of Accountants. Importantly, the CPA Program continues to address the needs

of employers and leads to a highly valued and globally recognised designation. CPA Australia continues to

apply the same rigorous standards of competence as has always been required to achieve the CPA Australia

designation.

The flexible learning focus ensures that those who successfully complete the CPA Program have much more

than a solid working knowledge of accounting, finance and international business. CPAs have initiative and

strong organisational and time management skills, and are positioned to be future leaders of the profession.

They are committed and motivated self-starters who are work-ready. Graduate and employer surveys continueto show extremely high levels of satisfaction with the skills developed through the CPA Program.

CPAs offer a unique combination of skills that drives employer demand for CPAs. They drive business success

through their technical expertise and ability to see the bigger picture and take a strategic view of business.

The Professional Qualifications Advisory Committee meets four times a year to provide management with

the best possible advice on the educational and training framework for members of CPA Australia so that

management may develop and monitor educational and training strategy relating to the professional programs.

This continual review ensures that the CPA Program remains at the forefront of international accounting

education programs.

All CPA Program learning materials are updated regularly. A Learning Management System with interactive

online learning support provides candidates with additional support in their studies. In addition to this Learning

Management System, a range of additional learning resources and tools are available to complement the

distance learning materials, including workshops, webinars, tuition providers, revision kits and passcards.

CPA Program exams are a comprehensive assessment of the learning objectives for each subject area. High

standards are maintained by processes that include expert monitoring at all stages, from the development of

the exam questions to the full statistical analysis of every question and every exam paper. The CPA Program is

supported by processes certified externally under ISO 9001:2008 as meeting international quality management

standards.

The CPA Program provides a challenging, rewarding and world-class program that ensures the high standards

of the profession are maintained. It is a benchmark for professional accountants and the foundation for lifelong

professional learning.


4/25

Contents

The CPA Program 1

Introduction 1

The Professional Qualifications Advisory

Committee 1

The objective of the CPA Program 2

Structure of the 2013 CPA Program 2

Foundation level 2

Professional level 3

Practical experience requirement 3

2013 CPA Program examsFoundation level 4

Introduction 4

Retake policy 5

Exam enrolments 5

Exam venues 5

Exam preparation process 6

Exam security process 6

Setting the passing standard 7

Standard Setting using the Modified

Bookmark method 7

Definition of a minimally competent candidate 7

Number and criteria of SMEs 7

Duration of standard setting exercise 7

Statistical equating methodology 8

Scaled scoring ranges 8

Equating of exam forms 8

2013 CPA Program examsProfessional level 9

Introduction 9

Exam enrolments 10

Exam venues 10

Exam preparation process 11

Exam quality process 12

Setting the passing standard 13

Maintaining the standard 13

Psychometric evaluation 13

Level of difficulty 13

Discrimination power 14

Distractor functioning 15

Reliability 15

Exam Policy Advisory Committee 16

Candidate performance 17

Foundation level 17

Professional level 17

Reporting of results 18

2013 committees and staff 20

Professional Qualifications Advisory

Committee Members 20

Exam Policy Advisory Committee Members 20

Special Consideration Committee Members 20

CPA Program Staff 21


5/25

1CPA PROGRAM EXAM REPORT 2013

The CPA Program

IntroductionThe CPA Program commenced in 1986, and is CPA Australias professional education program.

Completion of the CPA Program is required for candidates to advance to full membership and

CPA status. The CPA Program is designed to provide graduates with a solid base of technical

accounting knowledge, together with the broad business knowledge and skills required to

prepare them to become the strategic business leaders of the future.

CPA Australia has strict quality assurance processes in place to ensure the CPA Program

continues to be recognised as a leading professional accounting education program.

The Professional Qualifications Advisory Committee

The CPA Program has been developed by CPA Australia in consultation with senior academics

and industry experts to ensure that it provides a practical, up-to-date, postgraduate approach

to business studies.

To ensure the CPA Program remains at the forefront of professional business education,

the Professional Qualifications Advisory Committee advises on the educational framework

for members of CPA Australia. This ensures relevance and quality, so that the program

continues to meet the needs of members and business.

The Professional Qualifications Advisory Committee is made up of eight eminent individuals

from academia, industry, large firms and the public and not-for-profit sectors.

The Professional Qualifications Advisory Committee is chaired by Professor Kim Langfield-Smith,

FCPA, Pro Vice-Chancellor (Academic Performance) Monash University. A full list of members

is provided on page 20.

Key responsibilities of the Professional Qualifications Advisory Committee are to:

provide strategic oversight and quality assurance of the CPA Program, including ongoing

development and maintenance;

provide advice, direction and oversight to the design, curriculum, conduct, assessment

and quality assurance of the CPA Program;

provide advice on the development of the CPA Program, including its design,

curriculum, assessment and quality assurance so that it is positioned as the highest-

quality professional education program by professional accounting bodies and the

business community;

review and advise on all educational components of the CPA Program;

provide advice on the use of leading-edge technology to enhance and complement

the delivery and assessment of the CPA Program;

provide advice and recommendations in respect of the strategic position of the

CPA Program with undergraduate and postgraduate university education, the education

continuum and within the professional work environment; and

provide advice on the development of the practical experience requirement of the

CPA Program.


6/25


The objective of the CPA Program

The CPA Program provides graduates with a world-class, internationally recognised professional

qualification. The program is designed to enhance a CPAs competitive advantage for

leadership positions in finance, accounting and business in the private and public sectors.

Distinctively professional in orientation, the CPA Program is strategically integrated with other

tertiary education in a continuum of learning for career success. The CPA Program offers

an exceptionally high standard of quality and is recognised by ISO 9001 certification. It is

a rigorous, integrated education and experience program. CPA Program professional level

integrates postgraduate education with practical experience to provide opportunities to develop

technical and strategic knowledge and skills, and implement these in authentic situations.

As well as ensuring that graduate CPAs are well informed about the latest technical issues

and developments in the profession, the program is positioned within the broader context of

contemporary business. The professional level compulsory segments include emphasis on:

corporate governance, ethics and integrity;

development and implementation of business strategy; and

decision-making and leadership.

The CPA Program equips graduates with strong analytical and problem-solving capabilities

designed to add real value.

In 2013 there were over 27 000 candidates enrolled in the CPA Program professional level,with over 54 000 individual enrolments per annum. The CPA Program professional level is the

largest postgraduate program in Australia and possibly the largest postgraduate accounting

program in the world. The CPA Program is based on the core knowledge that all new

members have completed through their studies in accounting, commerce or business.

All professional level segments are prepared by authors drawn from industry, commerce,

the public sector and tertiary education. Segment outlines, learning objectives and related

materials are reviewed annually by CPA Australia technical staff and independent panels of

subject matter experts. The CPA Program utilises high-quality open-learning educational

resources including printed study material, online learning tasks and workshop materials.

Structure of the 2013 CPA ProgramThe CPA Program comprises two levels:

Foundation levelfoundation level exams test candidates knowledge of core accounting

knowledge required to satisfactorily enable them to undertake the professional level; and

Professional levelincorporates postgraduate educational units together with a practical

experience requirement.

Candidates must satisfy the core knowledge requirements of the foundation level and complete

the professional level education units and experience in order to attain the CPA designation.

Foundation level

The foundation level consists of eight exams designed to test candidates against a series of

learning objectives. The learning objectives are designed to incorporate focus on core knowledgeareas required for successful entrance into the professional level of the CPA Program.

The average foundation level candidate is required to complete three exams.

The foundation level exams offered are:

Foundations of Accounting

Accounting Concepts and Principles

Financial Accounting and Reporting

Fundamentals of Business Law

Economics and Markets

IT and Business Processes

Business Finance

Management Accounting

Foundation level candidates must complete all their required foundation level exams and

advance to Associate status within 10 years of their first foundation level enrolment.


7/25


Professional level

Candidates enrolling in the CPA Program professional level are required to pass six segments

and complete the Practical Experience Requirement to complete the CPA Program.

Each segment is one semester in duration with a recommended 150 hours of self-study.

The total study time for six segments is 900 hours.

Candidates must complete four compulsory and two elective segments. These candidates

must complete the following segments.

Three compulsory segments:

Ethics and Governance

Strategic Management Accounting

Financial Reporting

Two elective segments selected from the following:

Advanced Taxation, Singapore Taxation or Malaysia Taxation

Financial Risk Management

Advanced Audit and Assurance

Contemporary Business Issues

Capstone (compulsory):

Global Strategy and Leadership

Associate members joining CPA Australia from 1 July 2007 must complete the CPA Program

and advance to CPA status within six years of joining CPA Australia. Associates who joined

from 2004 to 30 June 2007 must complete the CPA Program within their required timeframe

and advance to CPA status within eight years of joining CPA Australia.

Further information on the CPA Program professional level segment offering is available on

the CPA Australia website.

Practical experience requirement

All candidates are required to complete three years of supervised practical experience in a

professional accounting, finance or relevant business role. The practical experience requirementis designed to complement the theory in the professional level segments. Candidates are

required to demonstrate competence in the technical, personal effectiveness, business and

leadership skill areas. A full member of an IFAC body will need to verify that a candidate has

successfully demonstrated the required skills.

There were over 18 000 Associate members enrolled in the practical experience requirement

through 2013. These members, on average, show consistently stronger results in the distance

education units than those who are not enrolled, with more than a 70 per cent higher success

rate in exams.


8/25


2013 CPA ProgramexamsFoundation

level

IntroductionApplicants for Associate membership of CPA Australia are required to undertake a Member

Entry Pathways Assessment. Each applicant is assessed against the core knowledge

requirements to commence the professional level of the CPA Program. Applicants who do

not meet all the core knowledge requirements are advised of the foundation level exams that

they must successfully complete to progress to the professional level of the CPA Program.

Candidates are not obliged to complete the exams with CPA Australia. They can complete

their required core knowledge at a higher education provider and apply for a reassessment

of their qualification.

All foundation level exams were offered continuously from 13 May through until 13 December.

This extended exam period allows greater flexibility for candidates, who can schedule their

exams at times to suit their work and lifestyle.

Foundation level exams are closed book. Each exam is of 3 hours and 15 minutes duration.

All foundation level exams consist of 100 multiple choice questions and are offered using

computer-based testing. All exams comprise a combination of scored and unscored items,

thus providing an opportunity for CPA Australia to confirm the statistical validity of an exam

item before it is scored. Questions are delivered randomly and results are available tocandidates on the day of their exam.

Foundation level exams:

Foundations of Accounting

Accounting Concepts and Principles

Financial Accounting and Reporting

Fundamentals of Business Law

Economics and Markets

IT and Business Processes

Business Finance

Management Accounting


9/25


Each foundation level exam has a set of learning objectives. Candidates are examined against

each of these learning objectives. Foundation level is not a distance education program,

nor a program of study in the same mode as the professional level, but is a series of exams

to test candidates knowledge and skills in the required core knowledge areas.

Success in the foundation level ensures candidates have met all the prerequisites to enable

them to enter the professional level with the same chance of success as those who have come

through with an accounting degree. The professional level builds on the knowledge examined

in the foundation level exams.

Retake policy

In 2012 CPA Australia introduced a retake policy for those candidates undertaking foundation

level exams. The retake policy limits the maximum number of times a candidate can retake the

same exam, and also the timeframes within which exam resits are permitted. This policy was

introduced for the benefit of those candidates for whom the foundation level may not be the

most suitable study method.

All candidates have a maximum of four attempts at any one exam, with a requirement for a

minimum of 30 days between exam sits. The required waiting period has been implemented

to ensure candidates allow adequate study time between exams. Candidates who are not

successful in their foundation level exams are provided advice on available learning support andpossible alternative arrangements to assist them in meeting the core knowledge requirements

to be able to commence the professional level.

Exam enrolmentsTotal foundation level enrolments are shown in Table 1.

Table 1

Foundation level enrolments

Exam

Total 2013

enrolments

Foundations of Accounting 411

Accounting Concepts and Principles 1 916

Financial Accounting and Reporting 1 063

Fundamentals of Business Law 1 465

Economics and Markets 260

IT and Business Processes 741

Business Finance 425

Management Accounting 553

Total: 6 834

Exam venuesFoundation level exams are conducted by Pearson Vue on behalf of CPA Australia. All exams

are conducted at computer testing centres operated by Pearson Vue or its authorised

representatives around the world. In 2013 there were a total of 6834 enrolments in foundation

level exams. The exams were offered in over 60 countries, at 165 exam locations, including

Israel, Kuwait and Kenya. Candidates are required to schedule their own exam date within

the relevant exam window, which provides flexibility for them to plan their study around work

and other commitments.


10/25


Figure 1

Exams delivered by country in 2013

Australia 62%

Vietnam 7%

Singapore 7%

Hong Kong 5%

Malaysia 4%

China 3%New Zealand 3%

Indonesia 1%

Other countries 8%

Exam preparation processCPA Program exam questions are written by experienced practitioners and academics who are

specialists in each subject area. Writers are contracted by Pearson Vue and undergo a training

program designed to ensure skills in the structure, design and analysis of multiple-choice

testing items. CPA Australia is involved in the design and management of the training sessions,

together with educational experts from Pearson Vue. Training is conducted using a variety of

delivery methods including face-to-face and webinars, thus ensuring ease of accessibility for

writers all over the world.

All items are reviewed by educational experts from Pearson Vue and CPA Australia to ensure

items meet the requirements for testing learning objectives, and testing the required levels of

knowledge. Detailed ongoing analysis is undertaken to ensure all items are performing as required

during the testing process. Each exam consists of a prescribed distribution of new and pre-tested

items. All new exam items are pre-tested to evaluate statistical reliability and performance,

and these items are not included for the purpose of a candidates score in the exam.

All exams are prepared in accordance with a prescribed weighting for each learning objective.

To protect the integrity of exam content, great care is taken to ensure that only those that

need to see exam content have access to it. File access is restricted, printing is kept to a

minimum and the distribution of all information is tracked.

Security audit documentation is maintained to keep track of the flow of documents

between Pearson VUE, CPA Australia and subject matter experts throughout each phase

of exam development.

Exam security processCandidates are required to present photographic and signatory identification before taking their

exams. An electronic signature and a digital photograph is captured and compared with the

identification previously provided. This biometric data is also used when the candidate first takes

a seat in the exam room. The candidates digital photograph appears on the screen and the

test administrator confirms the right candidate is sitting the right exam.


11/25


Setting the passing standard

Standard Setting using the Modified Bookmark method

The CPA Program foundation level uses the Modified Bookmark method1for setting the passing

standard for all exams. The passing standard is set on a triennial basis, with additional standard

setting undertaken as learning objectives change for each exam. Following the initial standard

setting process, all exams are pre-equated using statistical information related to the level of

competence required and level of item difficulty.

The Modified Bookmark method is a widely used item-centered method for setting a passing

score. In this method, the standard setting process is done after the results have been analysed.

The following steps are generally carried out during this method:

1 A group of subject matter experts (SMEs) for each exam is convened, forming the

Standard Setting Committee (the Committee).

2 The items are ordered from easiest to hardest based on their location (difficulty) on the

underlying measurement scale.

3 SMEs review items, beginning with the easiest item, and decide if a minimally competent

candidate has a 0.50 probability of answering the question correctly. The assumption is

that the easy items at the beginning of the scale will be marked as Yes and, as the judge

progresses to the harder items, there will be a transition point to the items being marked

as No.

4 The SMEs are asked to place a bookmark at a transition point where the majority ofresponses change from Yes to No.

5 There are usually multiple rounds of placing the bookmark, with discussions between

the SMEs between rounds.

6 The pass score is determined by averaging the item difficulties corresponding to the

bookmark locations.

7 Once the pass score has been determined, the percentage of candidates passing the score

will be reviewed to determine the preliminary impact of the pass-score setting exercise.

8 The result is then submitted to the Committee. If the result is not acceptable, the

Committee should nominate an acceptable pass percentage (informed by the knowledge

of the exercise) and then set a new preferred pass score.

9 The SMEs would then be asked to examine the items in the vicinity of this new pass score

to see whether or not they agree with the modified image of the minimally competent

candidate. If so, the new pass score is accepted for recommendation to the Committee;

if not, there would have to be a further round of discussions among the SMEs to reach

consensus on the refined image of the minimally competent candidate.

Definition of a minimally competent candidate

The description of the minimally competent candidate is developed during the standard setting

meeting using the course Learning Objectives and test specifications. For the purposes of this

standard setting exercise, a minimally competent candidate is defined as the least capable

candidate able to pass (at a minimum level) the CPA Australia professional level.

Number and criteria of SMEs

Each foundation level exam requires the attendance of between six and eight SMEs. SMEs areselected on the basis of meeting the following criteria:

1 Expertise in the subject matter.

2 Familiarity of undergraduate studies subject matter.

3 Knowledge of CPA Program professional level content.

Duration of standard setting exercise

The standard setting process for each foundation level exam is conducted over a full day.

Sessions are carried out in a face-to-face and virtual environment using WebEx and

teleconference technology.

1 Additional information on this method can be found in Cizek, G. J. (ed.) (2001), Setting Performance

Standards: Concepts, Methods, and Perspectives, Lawrence Erlbaum, Mahwah, New Jersey.


12/25


Statistical equating methodology

As new items are seeded in exams, they are calibrated onto the existing measurement scale

according to the Rasch Unidimensional Model. Once this has been done, it is possible to use

any of the calibrated items in the bank to generate an exam and the resulting measure will

locate a candidate along the scale. The cut-score is also located on the scale so it is possible to

compare the location of the candidate to the cut-score. If the candidate is located at or above

the cut-score then they have passed; if they are located below the cut-score, they will fail.

Scaled scoring ranges

Foundation level candidates receive a pass or fail mark in addition to a candidate feedback

report, which is a graphic to indicate a candidates relative strength or weakness against each

learning objective.

In Semester 1, 2011 CPA Australia introduced the use of scaled scoring for each exam.

In addition to receiving a pass or fail result, candidates receive their score and information

disclosing the score required to pass.

Table 2 demonstrates the score scales (the lowest possible score to the highest possible score)

and the passing score required for each foundation level exam. A different scale range is used

to easily identify each exam.

Table 2

Exam scale ranges

Exam scale ranges

Exam Lowest score

Required

passing score Highest score

Foundations of Accounting 250 320 375

Accounting Concepts and Principles 875 945 1000

Financial Accounting and Reporting 625 685 750Fundamentals of Business Law 375 430 500

Economics and Markets 125 190 250

IT and Business Processes 0 65 125

Business Finance 500 570 625

Management Accounting 750 820 875

The scaled score allows candidates to judge how their score relates to the pass mark.

The scale does not compare their score to other candidates, it is a unique view of their overall

performance in the exam.

Equating of exam forms

Item response theory is used to calibrate scores from two or more exam forms on the same

scale. Exam forms are drawn from the calibrated item bank and no item appears on an

exam before it has been trialled and equated to the benchmark scale. Through equating,

passing standards are set so that an equivalent level of proficiency is required to pass

different forms of the exam. Each candidate who is administered a set of exam items receives

a statistically equivalent exam: one that is neither statistically easier nor harder than any

other candidate.

All foundation level exams have been pre-equated from Semester 1, 2011. The use of

pre-equating allows candidates to receive their results on the day of their exam.


13/25


2013 CPA ProgramexamsProfessional

level

IntroductionExams were offered for each CPA Program professional level segment at the end of each

semester. The exam periods were between 29 April and 3 May 2013 for Semester 1,

and between 22 October and 25 October 2013 for Semester 2. Each exam was of 3 hours

and 15 minutes duration.

The exams for most elective segments consist of 100 per cent multiple-choice questions.The exams for compulsory segments in 2013 comprised a multiple-choice component

(worth approximately 7075 per cent) and a written-response component (worth approximately

2530 per cent). The exams for Singapore Taxation and Malaysia Taxation consisted of

written-response questions only.

The exam for the capstone segment Global Strategy and Leadershipconsisted of a written

component (worth 80 per cent) and a multiple-choice component (worth 20 per cent).

The written component was based on two case scenarios and an extended case study,

each of which had been made available to candidates before the exam. The questions were

based on the case facts given in the case scenarios and extended case study.

All professional level exams were open book and non-disclosed. Open book exams are exams

where candidates are generally permitted to bring notes and books of their choice into theexam. As candidates have access to such material, open book exams emphasise application

and analysis-type exam questions. Exams are open book because this more closely parallels

the conditions candidates are likely to experience in their working environment. The exam for

Singapore Taxation restricts the materials which may be brought into the exam to relevant

legislation only. In non-disclosed exams, questions and answers are confidential and they

are not published after the exam results are made available.


14/25


Exam enrolmentsThe total number of exam enrolments for the CPA Program professional level in 2013 was

61 114. Exam enrolments by country are shown in Figure 2.

Figure 2

Exam enrolments by country in 2013

Australia 69%

Vietnam 1%

China 5%

Hong Kong 8%

Malaysia 8%

New Zealand 1%

Singapore 5.5%

Fiji 0.5 %

Other countries 2%

Exam venues

DeakinPrime conducts exams each year on behalf of CPA Australia in over 300 venues

worldwide. DeakinPrime adheres to its own and CPA Australias strict quality requirements inthe selection of exam venues and supervisors. Matters such as the availability of adequate

parking and the provision of quiet and comfortable venues for candidates are considered.

Candidates were individually advised in writing of the exam date and venue for the CPA Program

professional level segment(s) they were enrolled in. Semester 1, 2013 had exams at 261 exam

locations. Of these, 43 per cent of all locations were within Australia and 57 per cent were

overseas. Some new and re-established overseas exam centres included Mozambique,

Armenia, Panama and new centres in India and China. Semester 2, 2013 had exams at

261 exam locations. Of these, 40 per cent of all locations were within Australia and 60 per cent

were overseas. In 2013 the major centres were in capital and provincial cities around Australia,

and Hong Kong, Kuala Lumpur and Singapore.


15/25


Figure 3

Distribution of exam enrolments

16 000

17 000

18 000

19 000

20 000

21 000

22 000

23 000

25 000

26 000

27 000

28 000

29 000

30 000

31 000

32 000

33 000

0

{

243

45

Sem. 1 Sem. 2

Total

50 617

{

28

085

Sem. 1 Sem. 2

Total

57 911

{

31

424

Sem. 1 Sem. 2

Total

64 357

15 000

24 000

{

30

545

Sem. 1 Sem. 2

Total

61 066

{

30

092

26

272

29

826

329

33

30

521

30

031

Sem. 1 Sem. 2

Total

60 123

20072008200920102011

{

Sem. 1 Sem. 2

Total

59 904

2012

29

416

30

488

{

Sem. 1 Sem. 2

Total

61 114

2013

30

515

30

599

Exam preparation processCPA Program professional level objective exam questions are written by experienced practitioners

and academics who are specialists in each segment area. A review panel analyses each

question to ensure they are clear and unambiguous, that answers are technically correct and

that the questions are examining the knowledge, skills and abilities that candidates should

have acquired from their study of the segment. Each semester CPA Program staff prepare the

objective question section of exam papers for each segment by drawing on the pool of availablequestions, using several key criteria. CPA Program staff ensure that the percentage of questions

in an exam that relate to each module (part of a segment) is consistent with the percentage

of study time recommended for that module. CPA Program staff also ensure that there is an

appropriate balance between knowledge, application and analysis questions. Table 3 illustrates

the template completed by the CPA Program staff to ensure the content validity of a typical

CPA Program segment. The constructed response questions in the exams for compulsory

segments, Ethics and Governance, Strategic Management Accounting and Financial Reporting,

and the capstone segment Global Strategy and Leadership, were prepared by academics and

practitioners, and reviewed by a panel of experts. Exams for Singapore Taxation and Malaysia

Taxation comprise 100 per cent constructed response questions. These exams are prepared by

practitioners and academics who are specialists within the relevant taxation systems. All exam

papers are moderated by a minimum of two expert members.


16/25


Table 3

Exam checklist

Module

number

Number of

questions

Number of

questions as

a percentage

of the exam

paper

Recommended

proportion of

study time (%)

Number of

knowledge

questions

Number of

application

questions

Number of

analysis

questions

1 2 2 2 1 1 0

2 9 10 10 4 4 1

3 11 14 12 0 8 3

4 7 9 8 0 4 3

5 15 19 20 2 5 8

6 6 8 7 0 3 3

7 7 9 10 0 5 2

8 6 8 8 1 3 2

9 4 5 6 1 1 2

10 5 6 6 1 3 1

11 5 6 7 1 1 3

12 3 4 4 1 1 1

Total 80 100 100 12 39 29

As a percentage of the exam paper: 15 49 36

Exam quality processAll printing and handling of exam papers was performed in locations which require secure

access and are under 24-hour video surveillance. All associated material used in the printing

process was securely stored and destroyed immediately after printing. Exam papers weretransported by a secure courier service and required a specified individuals confirmation of

delivery. Supervisors notified DeakinPrime on the receipt of exam materials and also upon the

despatch of completed exam papers. Candidates were required to present official photographic

identification at all exams. Candidates who were unable to provide photographic identification

were required to complete a statutory declaration that stated and confirmed their identity.

Exam answers to objective questions were double-marked by two separate computer-driven

optical scanning systems. The individual results from one system were then matched with the

results from the second system to ensure the validity of the scanning process. Comprehensive

audit procedures ensure the accuracy of the scanning process and comprehensive statistical

review techniques are employed to identify any anomalies in the results.

Exam answers to the constructed response section of the Ethics and Governance, Strategic

Management Accounting, Financial Reporting, and Global Strategy and Leadership paperswere marked by a panel comprising CPA Australia members (all of CPA or FCPA status) and

experts drawn from academia, commerce, industry and practice. A detailed marking grid was

completed for each candidate response.

The exam for Singapore Taxation was marked by a panel comprised of senior tax specialists

with experience in the Singapore tax profession. The exam for Malaysia Taxation was marked

by a panel comprised of senior tax specialists with experience in Malaysia tax.

The constructed response answers in Ethics and Governance, Strategic Management

Accounting, Financial Reporting and Global Strategy and Leadership were marked by markers

who received extensive briefings and marked to an agreed marking grid. Marking grids are

developed by the exam panel, including the Chief Examiners. They are detailed documents

that clearly outline the marks allocated to points in each constructed response question.Markers are guided and monitored by the Chief Examiners, CPA Australia technical experts

and DeakinPrime to ensure that marking is consistent. Statistical analysis is conducted on

each markers scores to ensure consistency of marking and reliability of marks. At least two


17/25


markers were involved in the assessment of each paper. Approximately 20 to 25 per cent of

exam papers that fall within a certain specified range are remarked by different markers from

those used in the original marking to ensure consistency and fairness. A number of papers are

also identified by statistical analysis for re-mark by the Chief Examiners.

Setting the passing standard

Since 1994, CPA Australia has used the Angoff method to establish passing standards forprofessional level exams. The Angoff method is widely used by many professional bodies in

determining pass standards. The American Institute of CPAs (AICPA), for example, adopted the

Angoff method for determining pass scores for the American CPA exam in 1997. The Angoff

method requires a panel of experts to estimate a target level of proficiency (pass mark) for a set

of exam questions or for a given exam paper.

A separate panel is used for each of the segments. Panel members must be knowledgeable

in the particular cognate area, and about the candidate population in general, in order to

form an assessment of the target level of proficiency required. CPA Australia technical staff,

industry experts and instructional designers form the requisite panels.

In light of the knowledge, skills and abilities required of candidates who achieve the target level

of proficiency, panel members are asked to individually review each question on a particular

exam paper. In reviewing the question, panel members must ask themselves what percentage

of adequately proficient candidates should answer the question correctly.

The target level of proficiency (pass mark) of an individual panel member is an average of

their individual question difficulties and the paper as a whole. The proficiency level (pass mark)

for most exams is in the range of 60 to 65 per cent. The proficiency level is determined

by statistical analysis and expert recommendation of the appropriate level of proficiency

or standard. This ensures that candidates are graded fairly from one semester to another.

The experts are representative of the membership and the profession; consequently,

the expectations are high. There may be instances where the proficiencies are higher.

Maintaining the standard

A statistical procedure known as equating is used to ensure that standards are maintained in

semesters where there has been no structural change to the educational material. The standard

set by the Angoff method is the basis of equating the future exams. Statistical equating ensures

that the passing score for the October 2013 exam paper represents the same ability as the

passing score in the May 2013 exam paper.

Psychometric evaluationThe psychometric quality of the CPA Program objective exams is evaluated using standard

statistical relationships used in classical testing theory, such as the relationships between

candidate performance on exam questions and their exam scores, and the relationships among

candidate scores on the exam questions themselves. The goal of the psychometric evaluation

is to assess how well each exam has differentiated between candidates of different abilitiesand to identify which questions should be reused, rewritten or discarded. In the psychometric

evaluation of multiple-choice questions, many psychometric characteristics can be evaluated.

Three of the more common are:

level of difficulty;

discrimination power; and

distractor functioning.

Level of difficulty

The level of difficulty of multiple-choice questions is determined by measuring the percentage

of candidates answering the question correctly. A question with moderate difficulty will better

distinguish between candidates of different abilities.

Table 4 shows the levels of difficulty of CPA Program segment exams for 2013.


18/25


Table 4

Level of difficulty

Number of

questions

Difficult

< 10%

Moderate

10%

90%

Less

difficult

> 90%

CPA examsMay 2013Advanced Audit and Assurance 90 0 90 0

Strategic Management Accounting* 60 0 60 0

Advanced Taxation 59 0 59 0

Financial Risk Management 70 0 66 4

Financial Reporting* 45 0 45 0

Contemporary Business Issues 79 0 78 1

Global Strategy and Leadership** 16

Ethics and Governance* 60 0 59 1

CPA examsOctober 2013Advanced Audit and Assurance 90 0 89 1

Strategic Management Accounting 59 0 59 0

Advanced Taxation 60 0 60 0

Financial Risk Management 70 0 68 2

Financial Reporting* 45 0 45 0

Contemporary Business Issues 80 0 76 4

Global Strategy and Leadership** 15

Ethics and Governance* 59 0 56 3

* Figures for Strategic Management Accounting, Financial Reporting and Ethics and Governance are only

for the multiple-choice components of the exam.

** Because of the restricted number of multiple-choice questions in the Global Strategy and Leadership

exam, no statistical analysis of Global Strategy and Leadership is included in this or subsequent tables.

Discrimination power

In a four-option or five-option multiple-choice exam, questions should be asked so that the

candidate scoring higher grades is more likely to answer the question correctly than a candidate

receiving lower grades.

The categories of discrimination power are described below.

Discriminating questions

Candidates receiving higher total exam scores are more likely to get the correct answer thancandidates with total lower scores.

Non-discriminating questions

Candidates with higher total exam scores are neither more nor less likely to get the correct

answer than candidates with lower total scores.

Reverse discriminating questions

Candidates with total lower exam scores are more likely to get the correct answer than

candidates with higher total scores.

The discrimination power of a question is measured by the point biserial correlation coefficient.

The discrimination power of each multiple-choice question is measured by correlating

candidates aggregate scores to the answers for each option. Ideally, a questions pointbiserial correlation coefficient should be 0.20 or above. Table 5 shows the discrimination

power of the CPA Program segments combined.


19/25


Table 5

Discrimination power

Discriminating Non-discriminating

Reverse

discriminating

Semester 1 92.0% 8.0% 0%

Semester 2 93.3% 7.6% 0%

Distractor functioning:

Semester 199.8 per cent were functioning

Semester 298.7 per cent were functioning

Distractor functioning

Ideally, all the incorrect answers (distractors) of a four-option or five-option multiple-choice

question should be plausible (functioning) to a candidate of lesser ability. A distractor is

commonly deemed to be not functioning when fewer than 1 per cent of candidates select

that distractor. In the May 2013 CPA Program exams, 99.8 per cent of all distractors were

functioning. In the October 2013 CPA Program exams, 98.7 per cent of all distractors

were functioning.

ReliabilityThe reliability of exam scores is calculated to determine the degree to which candidates

are likely to earn similar grades on exams with different but comparable questions covering

the same subject matter. The coefficient alpha is a measure of the internal consistency or

homogeneity of the scores on a test. The reliability coefficient can range in value from 0 to 1,

and it is desirable for the level of reliability to be greater than 0.80. CPA Program exams have

consistently been highly reliable over many years. This is due to the exam questions being

written and refined by experts, the use of expert judgment to inform proficiency standards,

and the re-testing of exam questions for equating purposes.

Table 6 shows the reliability levels of the CPA Program segment exams.

Table 6

Reliability

CPA exams May 2013 October 2013

Advanced Audit and Assurance 0.89 0.90

Strategic Management Accounting* 0.85 0.80

Advanced Taxation 0.89 0.87

Financial Risk Management 0.88 0.89

Financial Reporting* 0.82 0.82

Contemporary Business Issues 0.83 0.83

Ethics and Governance* 0.86 0.85

* Figures for Strategic Management Accounting, Financial Reporting and Ethics and Governance are only

for the multiple-choice components of the exams.


20/25


Exam Policy Advisory CommitteeThe Exam Policy Advisory Committee provides advice to CPA Australia on CPA Program exam

matters. It is composed of senior CPA Australia members with wide experience in education.

The Exam Policy Advisory Committee considers:

the procedures used for the development of exam questions and the setting of

CPA Program exam papers;

the standards of and the location of exam centres used in the CPA Program;

the methods used and the security provisions implemented for the printing, distribution andcollection of CPA Program exam papers;

the procedures used for the selection of, and information provided to, exam supervisors;

the procedures used and the timelines for the CPA Program exam grading process;

the statistical analysis provided on CPA Program candidate performance; and

the final recommendation on the proficiency level of each CPA Program exam.


21/25


Candidateperformance

Foundation levelCandidate results for foundation level exams are provided as pass or fail only. Table 7 shows

the percentage of candidates passing each exam in 2013.

Table 7

2013 exam results

Exam Fail Pass

Foundations of Accounting 55 45

Accounting Concepts and Principles 43 57

Financial Accounting and Reporting 46 54

Fundamentals of Business Law 31 69

Economics and Markets 48 52

IT and Business Processes 41 59

Business Finance 30 70

Management Accounting 26 74

Candidates sitting foundation level exams receive their results on the day of their exam.Each candidate receives a personal report showing their performance on the scale for the

relevant exam. Candidates also receive a report showing their individual performance against

each learning objective.

Professional levelIn accordance with CPA Australia Professional Qualifications Advisory Committee policy,

the following grades were used:

Fail

Pass

Credit

Distinction

High Distinction.

Table 8 shows the performance of candidates by grade for each CPA Program professional level

segment for both semesters in 2013.


22/25


Table 8

Semester 1, 2013 exam results

Total number

sitting exam*

(27 314)

HD

%

D

%

C

%

P

%

N

%

Advanced Audit and Assurance 2209 7.6 11.6 13.5 37.9 29.1Strategic Management Accounting 4977 3.3 10.4 14.1 38.8 33.0

Advanced Taxation 2177 6.3 11.7 12.9 39.2 29.4

Financial Risk Management 1978 4.7 10.0 15.4 39.2 30.5

Financial Reporting 4853 3.8 5.9 9.7 36.9 43.3

Contemporary Business Issues 1693 7.3 10.9 14.9 38.1 28.7

Global Strategy and Leadership 3685 2.2 5.2 12.0 47.5 33.2

Ethics and Governance 5299 5.8 11.9 16.1 37.6 27.6

Singapore Taxation 56 7.1 7.1 17.9 37.5 30.4

Malaysia Taxation 74 1.4 5.4 9.5 35.1 48.6

Semester 2, 2013 exam results

Total number

sitting exam*

(27 386)

HD

%

D

%

C

%

P

%

N

%

Advanced Audit and Assurance 2132 7.4 11.3 12.3 38.1 30.9

Strategic Management Accounting 4712 5.3 9.2 15.7 40.3 29.1

Advanced Taxation 2184 6.6 10.3 16.3 35.9 30.7

Financial Risk Management 1979 4.5 9.4 15.7 36.4 33.9

Financial Reporting 4824 5.1 6.6 10.2 37.0 40.6Contemporary Business Issues 1830 8.4 12.8 15.1 38.0 25.6

Global Strategy and Leadership 3989 5.1 10.4 17.0 46.0 21.4

Ethics and Governance 5315 6.8 11.0 15.9 37.8 27.8

Singapore Taxation 73 2.7 15.1 43.8 38.4

Malaysia Taxation 87 5.7 10.3 36.8 47.1

* Total number sitting exam includes candidates sitting for the Public Practice Programs.

Reporting of results

Candidate results were released via three systems: SMS, online and conventional mail. The SMS

and online results system relies on the use of individual Personal Identification Numbers (PINs)

supplied to candidates. Result release via SMS was offered for Semester 1 and Semester 2,

with approximately 8304 candidates electing to receive their results via SMS in Semester 1 and

9353 in Semester 2. Both the SMS and online results systems are supported by a telephone

help line. Official confirmation of results is sent by conventional mail. Candidates who sit an

exam can review online their Personal Analysis Letter (PAL), which identifies their performance

by topic on the exam.

In the May 2013 exams, the last exam was held on 3 May and the results were released by the

online and SMS results system on 14 June. In this time, 27 184 exam papers were collected

from 261 locations worldwide, 1 512 885 objective answers were scanned twice and any

differences reconciled, 23 307 candidates constructed responses were marked, psychometric

and equating reports were produced, the Exam Policy Advisory Committee convened and the

passing proficiency levels were established.


23/25


In the October 2013 exams, the last exam was held on 25 October and the results were

released by the online and SMS results system on 6 December. In this time, 27 226 exam

papers were collected from 252 locations worldwide, 1 503 424 objective answers were

scanned twice and any differences reconciled, 23 451 candidates constructed responses

were marked, psychometric and equating reports were produced, the Exam Policy Advisory

Committee convened and the passing proficiency levels were established.

The timing of results release allows candidates to review their enrolment decision before

the commencement of the enrolment period for the next semester. The turnaround for theCPA Program compares favourably with other national and international professional exams.


24/25


2013 committeesand staff

Professional Qualifications Advisory Committee MembersAssociate Professor Foo Yin Fah FCPA

Professor Zahirul Hogue FCPA

Mr Hari Iyer FCPA

Mr Neil Jackson FCPA

Professor Kim Langfield-Smith FCPA (Chair)

Ms Deborah Ong FCPA

Mr Warwick J. Spargo FCPA

Professor Kim Watty CPA

Exam Policy Advisory Committee MembersMr Ken Devos CPA

Dr Christine Jubb CPA (Chair)

Professor Louise Kloot FCPA

Associate Professor Denis Vinen FCPA

Special Consideration Committee MembersDr Mary Dunkley FCPA

Mr Greg Ellis CPA

Ms Lynette Ellis CPAMs Jane Hamilton CPA

Ms Dianne Harvey CPA (Chair)

Dr Christine Jubb CPA

Ms Fay Kairn CPA


25/25


CPA Program StaffMr Richard BrownGeneral Manager Professional Programs & Pathways

Ms Desley WardEducation Manager

Ms Kristy GradyProfessional Level Manager (maternity leave)

Ms Ange RiceFoundation Level Manager

Ms Alisa StephensQuality Management Consultant

Ms Alexandra WilsonProfessional Level Manager

Ms Suzannah AndrewsTechnical ConsultantMr Irwin BushnellTechnical Consultant

Ms Katarina BuzaninProfessional Level Coordinator

Ms Sarah ChinnickEducation Advisor

Ms Charlene DRosarioProfessional Level Coordinator

Ms Jan HaverfieldAdministration Executive

Ms Elise LiterskiProfessional Level Coordinator

Mr John NgiamTechnical Consultant

Mr Travis PembertonFoundation Level Coordinator

Ms Mary ShevlinFoundation Level Coordinator (maternity leave)

Mr Rohit SinghProfessional Programs Systems Coordinator

Ms Vammy TrinhAdministration Executive

Ms Belinda Zohrab-McConnellTechnical Advisor (maternity leave)

annual exam report 2013

Documents