Development of the new Swiss licensing examination for human medicine: from concept design to first result. !Sissel Guttormsena, Christine Beyelera, Raphael Bonvinb, Sabine Fellera, Christian Schirloc, Kai Schnabela, Tina Schurtera, Christoph Berendonka a University of Bern Faculty of Medicine, Insitute of Medical Education, Bern Switzerland

b University of Lausanne Faculty of Biology and Medicine, Lausanne Switzerland

c University of Zürich Faculty of Medicine, Zürich Switzerland Summary Due to the release of the new Federal Act on University Medical Professions, the Swiss national federal licencing exam for human medicine (FLE) had to be revised. This paper describes the process from concept design to the first results of the implementation of the new exam. The development process integrated all stakeholders of this venture. During this process questions related to the assessment aims, the assessment formats, the assessment dimensions, the exam content, and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound exam according to international standards, and thereby stating the expected knowledge and skills level by the end of the undergraduate medical education. Finally, a centrally managed and locally administered exam with a written ‘Multiple Choice Questions’ exam and a practical ‘Clinical Skills’ exam in the format of an OSCE were developed. The first two administrations of the new FLE show that the exam concept could be fully implemented as intended. The aspired psychometric indices were achieved and the results support the validity of the exam. Furthermore, possible future changes in the format or content shall be discussed. Key Words: Licencing-exam, Exam Formats, Review, Conceptual and Scientific Rational, MCQ, OSCE

1 Introduction Swiss medical education has a long tradition in promoting high quality and sustainable standards. This was also the aim, when due to recent political and legal decisions a new Swiss federal licencing examination (FLE) was developed and introduced in 2011. This paper presents the scientific rational and the conceptual issues of the development of this exam. The goals were ambitious as we aimed at developing a valid exam both in respect of psychometric quality and feasibility, specially tailored for the Swiss medical education system, all-embracing the medical and educational practice of five faculties and two languages. Next to its primary function as an instrument for quality control by the end of the undergraduate education, it was also clear that the exam would have implications also for the undergraduate medical curricula. The central impelling elements are presented below and explain the multifactorial character of the development process.

The Federal Act on University Medical Professions [1] addresses the quality of the medical professions and it warrants within Switzerland the free movement of labour persons with a medical degree and a Swiss diploma. By way of example, Article 14 holds that students shall possess elaborated knowledge, clinical skills as well as adequate social competencies by the end of their education. The law was enacted in September 2007, giving the medical schools a certain degree of autonomy for the conception of the undergraduate curriculum. Also the “Swiss Catalogue of Learning Objectives for Undergraduate Medical training” (SCLO) [2] sets the frame and specifies the objectives of the new FLE. The SCLO uses the framework of the CanMeds [3].

There are many stakeholders involved in the development and maintenance of an exam with this impact. The Federal Office for Public Health (FOPH) carried the responsibility for the legal and financial aspects of the FLE. The five Swiss faculties of medicine were actively

involved throughout the entire concept as well as content development processes. The Institute of Medical Education (IML), University of Bern provided expertise in the methodology of assessment and was committed to taking the lead in the central management of the FLE. The active process of developing the new FLE complying with this framework started in February of 2007 as a national steering committee was founded by the FOPH for the strategic and political framing of the FLE. Later, subordinate to the steering committee a project group with related exam specific sub-groups was in charge of the conceptual development and the operative planning and preparation of the specific exam format, content and logistics. In 2011, the steering committee was transformed into the national exam commission for human medicine, appointed by the Swiss Federal Council.

It was clear from the beginning, that the FLE should have a high value in respect to validity, reliability, effect on learning process, acceptance of assessment format, costs and practical feasibility [4]. A psychometric sound and valid exam is “fair” for the students, because it minimizes arbitrariness of the assessment outcome. The exam should be fair also in the sense that it reflects the content and learning objectives of the SCLO. The exam must also be defendable, i.e. juridical proof in the sense that it must keep up with appeals from candidates who fail the exam. In order to achieve these aims the development and the decision-making process was supported by a scientific discourse; literature and relevant research was studied, relevance for our aims, best practice - as well as applicability for the Swiss system were continuously evaluated. The following questions were landmarks for the scientific discourse for the initial development:

I. Which are the assessment aims? The federal licencing exam shall assess whether physicians by the end of the undergraduate education do possess the necessary basic knowledge, skills, and attitudes as described in the MedBG. How can these global aims be broken down to assessable objectives?

II. Which assessment formats are conceivable for a FLE? During the undergraduate education, the students are faced with many different assessment formats, written and oral exams, with practical and theoretical characteristics. Not all of the possible formats are feasible for an FLE.

III. Which assessment dimensions are relevant in order to assess interdisciplinary knowledge and skills?

IV. Due to the SCLO, the FLE follows an interdisciplinary paradigm described as “problems as starting point”. The candidates’ management and handling of open problems shall be addressed. The particular dimensions needs to be specified.

V. Under which conditions should the content development proceed in order to find common national foundation? A challenge for the new FLE was to develop coherent assessment content reflecting the training in all five faculties. The development process should reflect the common “lege-artis” of the learning objectives listed in the SCLO

VI. Which concessions can and must be made when implementing a national licensing exam? With our framework the negotiation of feasible assessment aims and formats was an important part of the development. Not all potential interesting ideas could be realised.

Hence, this paper is mainly devoted to the clarification of above questions. The development of the FLE proceeded in three main phases, which also map the main structure of this paper: 1) concept development, 2) content development and 3) implementation. The landmark questions were a common thread throughout these development phases.

2 Developing the exam concept !"# $%&'())*+,-+&./0()*(123*(-4*+,-41/1,-3**The overall conceptual aims for the FLE were to create a feasible exam according to international accepted assessment standards, and thereby setting a clear statement for the expected competence level by the end of the undergraduate medical education. Based on these initial reflections on the overall aims and on the legal aspects the national working group set the following conditions for the further development of the future licencing exam: First, the FLE will only cover deliberately defined aspects of the medical competencies. Second, these aspects comprise namely applied theoretical knowledge and practical skills. Third, the FLE shall be in the format of an objective, standardised exam according to international standards. Forth, the exam shall be centrally developed in two languages (German and French) but de-centrally administrated at the local sites. The question of the assessment aims (question I) and the assessment format (question II) are interrelated. The clarification of the assessment aims is guiding the selection of suitable assessment formats on one hand, and on the other hand, the selection of the assessment format defines the assessment aims that can be assessed with the given format. The decision for assessing both knowledge and practical skills, besides being indicated in the law [1], is also based on findings that the assessment of two aspects of competencies better predicts the candidates’ readiness to enter professional life and advanced training than the assessment of a single competence only [5, 6, 7]

!"! 5%()0(/1-6*7,8*/,*(33&33*(..)1&4*/7&,'&/1+()*9-,8)&46&*The assessment of knowledge with a written exam reflects that the graduates must achieve a solid foundation of factual as well as applied clinical knowledge. Applied theoretical knowledge for a sanctioning exam can be tested by multiple choice questions (MCQ) and short answer questions (SAQ). MCQ exams can be administered with low costs per candidate in case of high candidate numbers. Alternatively, with SAQs the candidates must actively formulate the answers, hence a common argument is that this is a more valid measure of existing knowledge [8]. However, this advantage is outbalanced by disadvantages of lower reliability per unit of testing time [9], lower reliability due to subjective factors from manual scoring of the answers [8] and higher cost in case of high numbers of candidates, due to the required manual evaluation of the responses [9, 10]..

The implementation of a MCQ-exam would promise continuation both in regard to the former national Federal Exam, in which the MCQ-format has been standard since more than 30 years. But also in regard to the undergraduate education, as students at all Swiss faculties gain much experience with this exam format. Many validated different MCQ types have emerged over the years, an overview can be found in Case & Swanson andKrebs [11, 12]. Common for the various MCQ types is that they offer a set of responses. The examinee is required to indicate the true and false answers in the set or to choose an answer from the set, as opposed to composing an answer actively [11, 13]. The evaluation of the various MCQ types convinced us that 2 types of MCQ questions promise high levels of validity and reliability: the one-best-answer out of 3-5 answers item (type A) [14-17] and the multiple true-false item (type Kprime) [16, 18].

Script concordance testing (SCT), being a written standardised test based on predefined set of selection alternatives, is an upcoming interesting alternative for classical written assessments. It probes a specific facet of clinical reasoning: the ability to interpret medical information under conditions of uncertainty [19]. Scoring reflects the degree of concordance of these judgments to those of a panel of reference experts [20]. During the initial development of the FLE this format was not considered because relevant experience among the faculties was lacking; moreover its implementation in a high-stakes exam is rarely investigated.

2.3 5%()0(/1-6*7,8*/,*(33&33*+)1-1+()*.'(+/1+()*391))3 The assessment of practical skills for a high stake exam is a true challenge due to the necessary effort in developing a standardized but at the same time realistic setting. Many less standardised assessment formats (e.g. unstructured (practical-) oral examinations (such as the long case [21]) often applied in undergraduate medical education were rejected mainly because such assessments cannot be standardised to the necessary extent. Unstructured oral exams do not promise the assessment quality required for a licencing exam, as the validity and the reliability is low [22-24]. Further, because the FLE is carried out only after having successfully completed undergraduate education, all longitudinal assessment formats (such as the Mini-CEX [25]) where repeated measures of performance over a longer period of time are collected and collated to a meaningful overall score could not be considered.!

Standardised assessment of practical skills is mostly carried out in the format of ‘Objective Structured Clinical Skills Exams’ (OSCE). This exam format has become a ‘gold standard’ in the context of high stake practical skills examinations and is characterised by the use of standardised patients (SPs) and models. This exam format comprises a circuit of stations in which candidates perform a series of different clinical tasks. Depending on the objective of each station, the tasks may include various clinical aspects such as to take a focused history, to perform a physical examination, to provide counselling to a patient, to derive accurate diagnostic hypothesis and to propose an appropriate management plan.

The OSCE-format was first described by Harden mid ’70 [26]. Ever since the OSCE has been adopted around the world and has stood the test of time [27]. Various studies have demonstrated that OSCE have good reliability coefficients [28, 29]. This has not only been shown for examinations at one single institution, also comparable levels of reliability were found across multi-site and multi-language administrations [30] [31]. More over, standardised patient based practical examinations have demonstrated predictive validity for clinical performance meaning that the scores achieved in an OSCE can predict the candidates’ performance in the real professional context. For example, Tamblyn and colleagues showed that scores on a CS-exam were significant predictors of competencies in consulting and prescribing in initial primary care practice [32]. Further, exam scores show a sustained relationship over 4-7 year with indices of preventive care, and acute and chronic disease management [33]. Even more important, low scores achieved on national Licencing exams predict complaints to medical regulatory authorities [34]. The documented quality of the OSCE-format has resulted in four large-scale certification and licensing examinations in Canada and the Unites States: - Objective structured clinical examination for family physicians in Quebec, Canada, since

1990 [35] - Objective structured clinical examination of the Medical Council of Canada Qualifying

Examination Part II, since 1992 [36] - Patient-based Clinical Skills Assessment for foreign medial graduates as part of the

United States Medical Licensing Examination [37], since 1998 [30] - Clinical Skill Examination by the National Board of Medical Examiners, since 2004 [38]. Based on the accumulated evidence and international experiences a pilot study was conducted to evaluate the feasibility of a standardised patient-based practical examination as a component of the Swiss licensing examination at the University of Geneva in 2003. This pilot demonstrated that the endeavour of a standardised patient-based practical examination is feasible in the Swiss context as well and its preliminary data suggested that scores achieved have good reliability coefficient [39].

The use of standardised patients (SPs) instead of real patients is a topic of recurrent debate. SPs for medical education were first described by Howard Barrows [40] and were first used for neurological examination [41].. Even though the use of real patients in the assessment of clinical skills has a long tradition and has also several advantages (they show real pathologies) compared to standardized patients [42], the use of real patients for a standardised licencing exam would impede the aims set for reliability and feasibility [43]. The use of SPs has been widely established since the early 70ies [44] and became common in Switzerland since 2002 [45]. The use of SPs for licensing exams in the US and Canada is well accepted and has evolved to be a standard [37, 46]

!": ;7,,31-6*/7&*&<(2*=,'2(/3*(-4*3.&+1=>1-6*/7&*(33&332&-/*(123**The decision was taken to implement two complimentary national formats: a MCQ exam and a clinical skills (CS) exam in the format of a standardised patient based practical examination. It was decided that these formats should be centrally developed and managed but de-centrally executed on all five sites. The operational assessment aims were specified accordingly: • The new MCQ exam aims at assessing interdisciplinary knowledge and its application in

a clinical context. In comparison the former national MCQ licensing exams as well as many of the undergraduate MCQ exams apply a disciplinary-related approach.

• Given that the goal of the federal licensing exam is to verify the candidates’ readiness to enter residency and hence practice under supervision, the aim of the CS exam is to assess whether the candidates can actually apply the necessary basic knowledge and skills. More precisely, the CS exam focuses on how candidates perform their clinical skills (e.g. take a history, conduct a physical examination) and how they communicate and interact with their patient.

The decision to conduct a written and a standardised practical examination have far reaching consequences, as a number of competencies outlined in the SCLO can not be tested in the FLE. For example, higher order competencies that go beyond the pure knowledge and skills (like acting professionally in real life setting) cannot be addressed in the FLE. As a consequence the assessment of professional behaviour in daily practice as well as longitudinal measures of performance are in the responsibility of the medical faculties (as describe in the MedBG) [1]. Regular accreditation of their study programs intends to support successful accomplishment of this assignment.

? @&%&),.1-6*/7&*&<(2*+,-/&-/*

3.1 Overall issues for the content development The questions III and IV above, regarding which assessment dimensions are relevant and the necessary conditions for content development are particularly addressed in this section. To ensure the FLE test content validity, a blueprint was set up with predefined criteria on which the MC questions and clinical cases were selected. It included the following two main dimensions and four secondary dimensions: • Problem as starting points listed in the SCLO [2] – A problem is defined as a symptom,

sign or test result of a patient with which the physician may be confronted. For example: A clinical case may be introduced to the candidate as follow: “You meet Mr. X in the emergency room, he complains about chest pain... Please take a focussed history and a physical exam.” Main criteria for inclusions in the exam are complaints that are either common or potentially dangerous and need to be reacted upon quickly.

• 7 roles of the physician as defined in the CanMEDS [3] model – Medical Expert, Communicator, Collaborator, Manager, Health Advocate, Scholar, and Professional. Whereas the main focus of the FLE of this dimension lies on the role of the Medical Expert (MCQ, CS) and Communicator (CS)

• Also four secondary dimensions were included: (I) Setting – inpatient, outpatient, (II) Type of care – Preventive, Emergency, Acute, Chronic Rehabilitation, Palliative Care; (IV) Age – child, adult, elderly, (IV) Gender – female, male, either

Depending on the exam format, MCQ or CS, different approaches were taken in the development of the final exam content. The development process described below is tailored to the quality and content requirements of the Swiss FLE and utilises the national distributed expertise. Below the experience from the two first implementations of the exam (2011 und 2012) are consolidated.

Both exam forms and the Swiss FLE project was presented and discussed in depth in an international workshop with experts from the National Board of Medical Examiners (USA)

and the Medical Council of Canada in August 2010. This workshop gave valuable inputs for the development and the design of a series of pilot studies at the faculties.

3.1 Multiple Choice Question Examination (MCQ) Assessment dimensions The MCQ exam focuses on measuring factual as well as the theoretical application of clinical knowledge. Medical practice was reflected through the characteristics of the questions challenging interdisciplinary problem-solving abilities. In order to support realistic application of clinical reasoning, rather than discipline related knowledge testing, the MCQ questions were presented by a description of a concrete patient case (so-called patient-vignette) [47]. Each patient-vignette reflected an aspect of each blueprint-dimension e.g. the best diagnostic procedure (role of the physician: medical expert) of a 10 year-old boy (age, gender), brought to the family doctor (setting) with a knee injury (problem as starting point, type of care), is asked. Development of MCQs The development of the MCQ exam content was approached through the deliberate emphasis of interfacultary participation. Experienced clinicians – representing all specialties and all five faculties - were invited to MCQ writing workshops. In these workshops the clinicians were introduced to the design principles of good quality MCQ questions. This implies that each question follows the structure of a patient vignette, is relevant for medical practice, and tests the application rather than rephrasing of knowledge. Equipped with this knowledge the clinicians wrote the questions, individually or during the workshops. Consensual validation process A multipart revision process aimed at checking the correctness of content, the relevance and the level of difficulty of each question according to general practice and the SCLO. This process included revisions by interfacultary and interdisciplinary groups of hospital-based clinicians as well as by representatives from general practitioners. Only questions passing successfully through the whole process were used in the FLE. Questions that did not reach consensus regarding content or did not meet the quality criteria were eliminated during this process, others were sent back to the authors with comments about how to improve them according to the requirements. The latter will re-join the revision process once revised. Standardisation Standardisation of content and procedures was important throughout the whole development process. First, the revision process described above is related to the standardisation. On the level of formal quality, standardisation was achieved by a central formal review of the questions. This took place in IML by assessment experts according to validated standards for MCQs [48]. The approved questions were at last translated in German and French respectively. To reduce variations among the translated versions, the same two translators carried out all translations. Both translators have a medical background and translated the cases into their native language. To ensure the congruency of the original and translated versions, medical professionals reviewed all questions. Second, all candidates received standardised information about the exam (to be found on the FOPH homepage (BAG, Internet). Furthermore the candidates were offered an on-line accessible set of self-assessment questions that are similar in respect to content, difficulty and time constrains to the questions used in the summative exam (IML, Internet). Third, each faculty received standardised, consolidated instructions for the organisation and execution of the exam Finally, an overall dimension of standardisation was given by the fact that the students performances were analysed by one single institution (IML) using standardised procedures of MCQ-exam analysis.

3.2 Clinical Skills Examination (CS) Assessment dimensions

Given the intent that the CS exam complements the written MCQ exam, the CS exam focuses on actual clinical performance and communication. The focus of the assessment within a patient-candidate encounter is placed on two main dimensions. The first dimension consists of content specific aspects like history taking, physical examination, diagnosis, management plan and counselling. The second dimension focuses on the communicative aspects as well as the candidates’ ability to engage in an empathic relationship. The assessment criteria for the first dimension were different for each case whereas the communication dimension was assessed across all cases with the same criteria. Development of the CS stations The case development process went through several steps and started with elaborating adequate and valid topics. Point of origin for the search for sensible topics was the 277 ‘problems as starting point’ from the SCLO. These topics had to fit the blueprint and needed to be playable in an encounter with standardised patients (SPs).

The second step implied the concrete case production. Once a specific topic was selected a content expert (senior clinician) at one of the five medical faculties was asked to write a case scenario. In a case development workshop this preliminary case scenario was further elaborated with help of a second clinician (from a different discipline and different faculty) and a CS coach (a medical educationalist who has in depth knowledge and experience with the CS test format). In these workshops not only the case content per se was developed but also how the different aspect of the content should be weighted against each other. The teamwork between two experienced clinicians (from two different faculties) and the CS coach guarantied that the quality aspects from the content perspective (does the case test relevant medical knowledge and skills?) as well as the quality aspects from the test format perspective (is the case playable within a 13 minutes simulated patient-candidate encounter?) could be reached. Also, the degree to which the cases were “playable” was tested during the case development workshops. The cases were role played with a SP at the workshops as soon as the teams developed a case. This gave early inputs regarding feasibility and showed necessary corrections immediately.

Consensual validation process The third step in the case development process comprised the consensus finding between the five medical faculties. Once the cases were finally drafted a national board consisting of members from all five medical faculties and representatives from general practitioners reviewed the cases. The main tasks of the review board were to ensure that the developed test content is relevant in respect to the skills and knowledge deemed to be necessary to function properly as a physician, second to ensure that the degree of complexity matches the educational level of the candidates and third to ensure that the test content is accessible and thus taught in a comparable way at all five sites. According to the decision of the review board the cases were either accepted to be included in the CS exam, sent for revision or declined. The validated cases were handed over to the of SP trainers. Standardisation As a fourth step all cases were trained centrally with SP trainers from all the medical faculties. The SP trainers had to agree about exactly how the different roles had to be portrayed. For challenging roles videos were produced to clarify for example the extent of a neurological deficit to be portrayed. The approved final cases were at last translated in German and French respectively. The translation proceeded as described for the MCQ.

As a last step before the exam took place, standardized information and training meetings were held for the examiner at all five sites highlighting the role and duties of the rater and how to rate the communication dimension. The candidates received standardised instructions (written as well as video examples) containing general information about the exam. Finally the candidates’ performances were analysed centrally by one single institution (IML) using standardised procedures.

4 Implementation !Preliminary versions of the exams were pilot tested in 2009/2010 and the knowledge gained has been integrated in the discussion related to the scientific questions. Computer-Based Assessment (CBA) with multimedia rich MCQs were implemented on a pilot basis for the first exam. Due to the focus set for this paper, the issues related to CBA have not been treated here. .!

:"# A2.)&2&-/(/1,-*,=*/7&*B;C*&<(2*The MCQ exam was administrated locally and simultaneously at the 5 sites in 2 sessions separated by one day. The two sessions lasted 4! hours each and were separated by one day. Each session contained 150 questions covering potentially all dimensions of the blueprint. The total number of 300 questions is regarded to be necessary for sufficiently covering the content (as described in the SCLO and in the blueprint) and for a reliable measure of the candidates' knowledge (calculated on the basis of the former Swiss FLE and the Spearman Brown prophecy formula). The implemented 300 questions is in line with international standards (Licensing Examination of the USA (USMLE, Internet) and the Qualifying Examination of Canada (MCCQE Internet). With 108 seconds to answer each question, the Swiss MCQ was considering offering enough time per question, considering the 90 seconds per question in the USMLE [15].

:"! A2.)&2&-/(/1,-*,=*/7&*;D*&<(2*The CS exam was administered locally at the 5 sites. Depending on the number of candidates at the different sites the exam was administered over 2-4 consecutive days. The individual candidate was assessed in twelve stations, each with a 13-minutes patient encounter session. Each exam day consisted of a different set of 12 stations, adding up to a total number of 48 stations. All sites testing the same day used the same set of 12 stations. Each candidate was scheduled for one 3 hours and 45 minutes exam session (including three 15-minutes breaks and 2 minutes rotation time in-between the different stations). A total of 58 exam sessions with up to 14 students scheduled per session were administered. This implies that simultaneously 2 to 3 sessions ran parallel at the each site during the 4 testing days.

Experienced clinicians who were recruited form the local faculty rated the performance of the candidate during the patient encounters. The aspects of history taking, physical examination as well as the differential diagnosis and management plan (subsumed under the term ASM) were rated with a case specific checklist. Whereas the communication skills (subsumed under the term KK) were rated with a uniform generic four-dimensional scale adapted from [49, 50]. All examiners who participated for the first time in the new federal licensing examination had to participate in an orientation meeting and training session that highlighted the execution and scoring process of the CS exam.

5 Overall outcome E"# F&-&'()*1-=,'2(/1,-*The exams in 2011 and 2012 were comparable in respect to format, content and results, hence, detailed information is provided for the 2012 exam only: The number of candidates from the 5 Swiss faculties was 784 (MCQ) respectively 785 (CS). The number of candidates per faculty varied between approximately 120 and 240. Approximately 70 % of the candidates took the exam in the three German-speaking faculties and 30 % in the two French-speaking faculties. Additionally, 43 (MCQ) resp. 16 candidates (CS) with a medical degree from countries outside EU/EFTA attended the exam with the purpose of obtaining a Swiss medical diploma.

Results were calculated individually for all the candidates, both Swiss graduating students and external candidates for both exams. The exam committee was mandated to

finally set the pass/fail limit based on the calculations performed with the exam related methods. Below overall descriptive data for the two exams is presented.

E"! $%&'())*,0/+,2&3*B;*5<(2*The scores of the two exam sessions were totalled into complete scores per student. Only this total score was considered for the pass/fail decision. Table 1 shows the mean percentage score and standard deviation of the exam as well as the reliability index (Cronbach Alpha) [51]. Table 1: Key Results for the MCQ exam 2012

Mean correct answers 73.8 % Standard deviation 6.6 % Cronbach Alpha 0.91

The national exam committee defined the passing score on the basis of two content related methods [52, 53]. Whereas the pass rate of candidates of the 5 Swiss faculties was high (between 96.8 to 100%) only 67.4% of the foreign medical graduates passed the MCQ examination. The results were homogenous between the faculties.

E"? $%&'())*,0/+,2&3*;)1-1+()*391))3*&<(2*In a first step test item difficulties were calculated and comments from examiners and candidates were carefully processed. Based on these results the reliability indices were computed (Cronbach alpha between 0.86 and 0.90 for the individual exam days).

Table 2 shows the means, standard deviations, minimum and maximum of the total exam score as well as the ASM and KK component scores collated over the four exam days. The total score is a composed weighted score (total score = 0.75*ASM + 0.25 * KK). Overall the scores assessed a wide range of performance. However the scores achieved in the ASM component were consistently and substantially lower compared to the KK scores component. Furthermore, there were small but significant differences in the performance scores over the four days. To account for this ‘day effect’, the total scores were adjusted to a common mean (z-transformation). In a next step, the borderline regression method was applied to calculate the passing score [54].

Table 2: Means, SD, Minimum and Maximum in Percentage of the Total exam score, ASM and KK component scores (results based on all 801 candidates) Mean (%) SD (%) Minimum (%) Maximum (%)

Total score 73.0 6.1 44.8 87.9 ASM score 69.0 6.7 38.2 86.5 KK score 85.0 6.4 47.5 98.3

The national exam committee used the calculated results as a basis for their decision. The pass rate of candidates of the 5 Swiss faculties was high (between 97.5 to 99.2%). In contrast, in the foreign medical graduates group passed 50% of the candidates the CS exam. Thereby it is important to note that the foreign medical graduates did not fail the CS exam because of insufficient communication (language) skills but because of low scores achieved in the ASM component indicating a lack of applied medical knowledge and skills.

E": A-/&'G+,''&)(/1,-*H&/8&&-*;D*+,2.,-&-/3*3+,'&3*(-4*H&/8&&-*/7&*;D*(-4*B;C*&<(23*

Pearson correlation was used to determine the relationships between MC score and CS overall as well as the CS component scores. The moderate correlation (r = 0.52) between the CS and MCQ exam score indicates that the two exams measure separate and distinguished competencies (that of course share a common ground) but are complementary in assessing the candidates’ competencies. Furthermore, the low correlation between the KK component of the CS exam and the MC scores (r = 0.36) versus the moderate correlations

between the ASM component of the CS and the MC scores (r= 0.51) indicate construct validity of the two assessment formats. In other words, these correlation matches with the expectation that the content related aspects of e.g. history taking has more in common with the type of knowledge that is evaluated in an MCQ. On the other hand, and as expected, the ability to engage in an empathic patient relation (KK) correlates weakly with factual knowledge (MCQ scores).

I @13+0331,-*(-4*+,-+)031,-3* The aims for the new FLE were to develop a high stake exam for the Swiss medical faculties, comprising to the legal framework as well as to scientific and best evidence assessment standards. Due to its composition and form, the exam sets the standard of which levels of knowledge and skills are to be achieved by the point of graduation. Both applied clinical knowledge as well as communicative competencies is assessed. In this context, it is important to state that the new FLE should not replace, but set the end point of a series of intra-curricular exams, and thereby constitute a minimal performance standard for the end of the undergraduate education. We have formulated the multipart challenges for this development process as five driving questions. This paper addresses the rational and solutions found for these questions and thereby also the conceptual foundation for the new Swiss FLE.

Although the overall assessment aims were given in the new law, the deployment of the legal framework into operational aims, namely into a feasible and valid exam, was a long process and demanded close cooperation between assessment experts, medical content experts and the involved decision makers (medical faculties and FOPH).

Of the many existing assessment formats only few qualify for a standardised national licensing examination. The formats under consideration had to be known to and accepted by the five faculties, as well as feasible both in relation to financial and practical terms. Last but not least the selected assessment formats should have a record of convincing empirical evidence. The mentioned aspects as well as circumstances related to practical feasibility reduced the possible assessment formats to the two described above. Taking the overall aims and our experience with the first exams into considerations, the two formats implemented lived up to the initial expectations.

In order to specify the assessment dimensions it was necessary to deliberately define the competences that are deemed to be necessary to function properly as a physician. The operational description of the medical actions that candidates must undertake to solve a given problem was a major part of the content development. The processing of MC questions and CS cases should accurately reflect the candidates’ knowledge and skills to handle medical problems that are encountered in the real world. An important related aspect was to simulate the reality of medical situations in the exam setting (particularly for the CS exam) in order to induce a minimal degree of immersion into the settings by the students (ecological validity). The predictive validity of the exam formats is documented above, hence the exam setting was built based on the following proposition: Candidates who are not capable to perform to a minimal level in the simulated exam setting lack essential basic knowledge and skills and can not be expected to function properly in the authentic setting.

The question addressing how to manage the content development process was a key aspect. It was clear that the different curricula from the five medical faculties must be represented in the FLE. A part of the solution was to bring experts/clinicians from all the faculties together for the content development process. These meetings took place face to face and made fruitful and agile discussions possible. We regard the deliberate process for consensus finding as an important aspect of the successful implementation. Starting point for many discussions was the Swiss catalogue of learning objectives (SCLO). Despite the fact that some limitations of the SCLO became evident, also many debates related to desired relevance and depth of particular expert issues could be avoided. In this sense, the SCLO, which was not an issue of debate on this stage, took the function of a mediating tool for the content development because it regulated which issues to include in the FLE. Parallel with

the content development process the educational deans met regularly with the steering group and medical assessment experts. In this way the necessary acceptance for the product was in line with the development progress. Questions could be discussed regularly, and the ownership of the faculties for the developing concept could be continuously fostered.

In order to develop a standardised national licensing exam some concessions had to be made. The new FLE may give the impression of having sacrificed authenticity and flexibility for good statistical values. According to the operational goals for the FLE the exam meets the expectations. However, concessions were necessary regarding the lack of physical or morphological pathologies, which SPs do not have (absence of pathological findings e.g. heart/lounge murmurs. Moreover, it is evident that babies and small children cannot be portrayed by SPs, mainly for ethical reasons. As a consequence not all necessary medical competences can be assessed in the FLE. However, as indicated in the law, the FLE does by no means replace the responsibility of the universities to cover the various educational aspects during the education. Therefore, situated learning [55] and assessment in the authentic setting has to (and can only) take place during undergraduate education. Hence, it is of great importance that the faculties develop meaningful programmes of assessment complementing the teaching and learning during undergraduate education. Further, as the FLE should be a product under continuous development, concessions in the first phase can be adapted with more amendable solutions throughout the further development of the FLE. Aspects being under consideration are issues such as advanced clinical reasoning strategies and the inclusion of more (real) pathological findings in the FLE. In the first case script concordance testing (SCT), as mentioned above, offer a promising format whereas high fidelity simulations / computer based assessment (CBA) carry the potential to present pathologies with sounds and video (heart murmurs, wheezing in a toddler, gait disorder in an elderly patient).

The leading aims for the FLE were to create a feasible, fair, valid and psychometrically sound exam, and thereby setting a clear statement for the expected knowledge and skills level by the end of the education. After more than 4 years of development the proof of concept was the first implementations of the exam. The introduction of the FLE showed that thorough planning and the work invested on quality assurance were worth the effort. The FLE could be fully implemented. Both the new national MCQ and the coordinated simultaneous execution of a standardised patient-based practical examination at all five sites in two languages were possible. Moreover, the achieved psychometric indices are more than satisfactorily and the results support the construct validity of the two assessment formats.

These results are encouraging and a good basis for further development. The continued maintenance and development of the FLE is an inherent aspect of sustainable quality management. Following the scientific paradigm possible effects of future changes in the format or content shall be traced with data and continuously empirically validated.

J+9-,8)&46&2&-/3*!"#$"%&'()'*+"',-*.(,-/'01-#'2(##.&&.(,')(%'*+"'3"4"%-/'01-#'56788'9'678:;'!"#$"%&'()'*+"',-*.(,-/'<=*""%.,>'>%(?@'3"4"%-/'A.B",&.,>'01-#.,-*.(,C'5677D'9'6788;'!"#$"%&'()'*+"',-*.(,-/'<E%(F"B*'>%(?@'3"4"%-/'A.B",&.,>'01-#.,-*.(,C'5677G'9'6788;'!"#$"%&'()'*+"',-*.(,-/'<H(%I.,>&'>%(?@'!2JC'-,4'*+"'<H(%I.,>'>%(?@'2=C'5677G'9'678:;'K//'*+"'"1@"%*&'L+('B(,*%.$?*"4'*('*+"'B(,*",*'4"M"/(@#",*'-,4'-4#.,.&*%-*.(,'()'*+"'"1-#&'

K13/*,=*(HH'&%1(/1,-3**!

ASM Assessment Status Management (The aspects of history taking, physical examination as well as the differential diagnosis and management plan)

CBA Computer-Based Assessment CS Clinical Skills Examination FLE Swiss federal licencing examination FOPH Federal Office for Public Health

MCQ Multiple Choice Questions MCCQE Medical Council of Canada Qualifying Examination MedBG Medizinalberufegesetz. The Federal Act on University Medical Professions Mini-CEX Mini-Clinical Examination NBME National Board of Medical Examiners, USA KK Communication Competences / Kommunikationskompetenzen OSCE Objective Structured Clinical Skills Exams IML Institute of Medical Education; Medical Faculty, University of Bern SAQ Short answer questions SCT Script concordance testing SCLO Swiss Catalogue of Learning Objectives for Undergraduate Medical training SP Standardised Patient USMLE United States Medical Licensing Examination

References "#! $%&'(#!!"4NOP'5677Q;P'O?,4"&>"&"*R'S$"%'4."'?,.M"%&.*T%"'!"4.R.,-/$"%?)"P'

)**+,--...#/&012#3)-3)-&-/4-5667-869"#+&:#!566;!!<31*%&!56"9!";#68#56"9=>!?@/1A/BA%!:CD0,!)**+,--...#/&012#3)-3)-&-/4-5667-869"#+&:#!

5#! EFGH#!=L.&&'2-*-/(>?"'()'A"-%,.,>'U$F"B*.M"&')(%'V,4"%>%-4?-*"'!"4.B-/'W%-.,.,>#!566I!!<31*%&!56"9!";#68#56"9=>!?@/1A/BA%!:CD0,!)**+,--43AD#401:J#3)#!

9#! KC/2JL!M#L!%&#!W+"'2-,!0X='677Y'@+Z&.B.-,'B(#@"*",BZ')%-#"L(%IP'O"**"%'&*-,4-%4&P'O"**"%'@+Z&.B.-,&P'O"**"%'B-%"P'#!566NL!O)%!PDQ/A!FDAA%R%!D:!S)Q4131/24!/2&!ETCR%D24!D:!F/2/&/,!H**/./#!

8#! U/2!V%C!UA%T*%2L!F#S#$#L!W+"'-&&"&&#",*'()'@%()"&&.(,-/'B(#@"*",B"['X"M"/(@#",*&\'%"&"-%B+'-,4'@%-B*.B-/'.#@/.B-*.(,&P!?&@/23%4!12!W%/A*)!E31%23%4!X&T3/*1D2L!"YY;#!!Z"[,!+#!8"\;7!!

N#! ]1AJ124D2L!O#M#!/2&!F#$#!KC/0+*D2L!2(#@%"+",&.M"'?,4"%>%-4?-*"'#"4.B-/'-&&"&&#",*&'.#@%(M"'@%"4.B*.(,'()'B/.,.B-/'@"%)(%#-,B"P!$%&!X&T3L!5668#!"#Z"6[,!+#!""""\;#!

;#! W/0&QL!W#L!SC/4/&L!^#L!?2&%C4D2L!$#!'#L!E3)%C+B1%CL!?#L!]1AA1/04L!P#L!_.1%C4*C/L!P#L!FT&&1)QL!W#L!O0!0'&Z&*"#-*.B'%"M."L['E%"4.B*.M"'M-/?"&'()'#"-&?%"#",*&'($*-.,"4'.,'#"4.B-/'&B+((/&'-,4')?*?%"'@"%)(%#-,B"'.,'#"4.B-/'@%-B*.B"P'P!$%&13/A!O%/3)%CL!566;#!$#Z5[,!+#!"69!`!"";#!

7#! ]%2R)D:%CL!X#L!%*!/A#L!X(B*(%'&B(%"&'(,',-*.(,-/']?-/.)Z.,>'"1-#.,-*.(,&'@%"4.B*']?-/.*Z'()'B-%"'.,')?*?%"'@%-B*.B"P!$%&13/A!X&T3/*1D2L!566Y#!%",!+#!"";;\""79#!

I#! ]%40/2L!?#(#L!H%.*.,>'*+"'W"&*'^*"#L!12!04?B-*.(,-/'!"-&?%"#",*P'H-&+.,>*(,L!P#G#!O)DC2&1J%L!X&1*DC#!"Y7"L!?0%C13/2!FDT231A!D2!X&T3/*1D2!+#!I"\"5Y#!

Y#! '%22%**L!P#X#L!U,'*+"'!"-,.,>&'()'2(,&*%?B*"4'_"&@(,&"L!12!2(,&*%?B*.(,'`"%&?&'2+(.B"'^,'K&&"&&#",*L!P#X#!'%22%**!/2&!]#F#!]/C&L!X&1*DC4#!"YY9L!W1AA4&/A%L!a%.!M%C4%Q,!G/.C%23%!XCAB/T0!?44D31/*%4#!+#!"\57#!

"6#! E3)T.1C*)L!G#]#O#!/2&!F#S#$#!U/2!&%C!UA%T*%2L!KO2'()'/"-%,.,>'-,4'*"-B+.,>'.,'#"4.B.,"['H%.**",'-&&"&&#",*P''P!'$M!5669#!"$&Z79Y6[,!+#!;89\8N#!

""#! F/4%L!E#!/2&!V#'#!E./24D2L!2(,&*%?B*.,>'H%.**",'W"&*'J?"&*.(,&')(%'*+"'O-&.B'-,4'2/.,.B-/'=B.",B"&L!"YYIL!a/*1D2/A!'D/C&!D:!$%&13/A!Xb/012%C4,!S)1A/&%A+)1/#!

"5#! ^C%B4L!P#!K,/".*?,>'R?%'a"%&*"//?,>'M(,'!2b3%->",'?,4'!2bE%S)?,>",')S%'4."'T%R*/.B+"'K?&$./4?,>#!5668>!?@/1A/BA%!:CD0,!)**+,--...#10A#T21B%#3)-&1%24*A%14*T2R-/44%440%2*c+CT%:T2R%2-#!

"9#! OC/TBL!P#X#L!U,'*+"'0]?.M-/",B"'()'*+"'W%-.*&'K&&"&&"4'$Z'!?/*.@/"b2+(.B"'-,4'2(,&*%?B*"4b_"&@(,&"'W"&*&L!12!2(,&*%?B*.(,'`"%&?&'2+(.B"'^,'K&&"&&#",*PL!P#X#!

'%22%**!/2&!]#F#!]/C&L!X&1*DC4#!"YY9L!G/.C%23%!XCAB/T0!?44D31/*%4,!W1AA4&/A%L!a%.!M%C4%Q#!+#!"\57#!

"8#! E./24D2L!V#'#L!%*!/A#L!E&ZB+(#"*%.B'B+-%-B*"%.&*.B&'-,4'%"&@(,&"'*.#"&')(%'B(,*",*b@-%-//"/'"1*",4"4b#-*B+.,>'-,4'(,"b$"&*b-,&L"%'.*"#&'.,'%"/-*.(,'*(',?#$"%'()'(@*.(,&P!?3/&!$%&L!566;#!#!Z"6!ET++A[,!+#!EN5\N#!

"N#! E./24D2L!V#'#L!^#_#!WDA*d0/2L!/2&!^#!?AAB%%L!!"-&?%"#",*'B+-%-B*"%.&*.B&'()'B(,*",*b@-%-//"/'&.,>/"b$"&*b-,&L"%'-,4'"1*",4"4b#-*B+.,>']?"&*.(,&'.,'%"/-*.(,'*(',?#$"%'-,4'&(?%B"'()'(@*.(,&P!?3/&!$%&L!566I#!#"Z"6!ET++A[,!+#!E5"\8#!

";#! PDR/T43)L!?#L!P#!WD:%CL!/2&!P#!^C%B4L!_-%"/Z'&"/"B*"4'4.&*%-B*(%&'.,'+.>+'&*-I"&'#"4.B-/'#?/*.@/"bB+(.B"'"1-#.,-*.(,&'-,4'*+".%'%"B(>,.*.(,'$Z'.*"#'-?*+(%&['-'&.#?/-*.(,'-,4'&?%M"ZP!'$F!$%&!X&T3L!56"6#!!'ZIN[#!

"7#! PD&C1RT%dL!$#F#L!W+%""'(@*.(,&'-%"'(@*.#-/')(%'#?/*.@/"bB+(.B"'.*"#&['-'#"*-b-,-/Z&.&'()'G7'Z"-%&'()'%"&"-%B+P!M!X&T3!$%/4,!e44T%4!/2&!SC/3*13%L!566N,!+#!9\"9#!

"I#! ^C%B4L!P#L!W+"'=L.&&'H-Z'*('=B(%"'!?/*.@/"'W%?"b3-/&"'^*"#&['W+"(%"*.B-/'-,4'0#@.%.B-/'0M.4",B"L!12!K4M-,B"&'.,'!"4.B-/'04?B-*.(,L!?#M#M#?#!E3)C%+B1%CL!%*!/A#L!X&1*DC4#!"YY7L!^AT.%C!?3/&%013!STBA14)%C4,!VDC&C%3)*#!+#!"NI\;"#!

"Y#! GTB/C4JQL!E#L!%*!/A#L!=B%.@*'B(,B(%4-,B"'*"&*.,>['3%(#'*+"(%Z'*('@%-B*.B"['K!00'N?.4"'c(P'DYP!$%&!O%/3)L!56"9#!"(Z9[,!+#!"I8\Y9#!

56#! F)/CA12L!'#L!%*!/A#L!W+"'=B%.@*'2(,B(%4-,B"'*"&*['-'*((/'*('-&&"&&'*+"'%")/"B*.M"'B/.,.B.-,P!O%/3)!G%/C2!$%&L!5666#!!$Z8[,!+#!"IY\YN#!

5"#! aDC3121L!M#M#L!W+"'4"-*+'()'*+"'/(,>'B-&"d!'$ML!5665#!"$%Z7998[,!+#!86I\Y#!55#! $Tdd12L!G#M#!/2&!G#!W/C*L!U%-/'"1-#.,-*.(,&L!12!K&&"&&.,>'2/.,.B-/'2(#@"*",B"L!U#P#!

a%T:%A&!/2&!(#P#!aDC0/2L!X&1*DC4#!"YINL!E+C12R%C!STBA14)12R!FD#,!a%.!fDCJ#!+#!7"\Y9#!

59#! E./24D2L!V#'#L!K'#"-&?%"#",*')%-#"L(%I')(%'@"%)(%#-,B"'$-&"4'*"&*&L!12!3?%*+"%'4"M"/(@#",*&'.,'-&&"&&.,>'B/.,.B-/'B(#@"*",B"P'L!e#P#!W/C*!/2&!P#$#!W/C&%2L!X&1*DC4#!"YI7L!F/2\W%/A,!$D2*C%/A#!+#!"9\8N#!

58#! ]/44L!U#L!U/2!&%C!UA%T*%2L!F#L!E)/*d%CL!M#L!MD2%4L!P#!L!K&&"&&#",*'()'B/.,.B-/'B(#@"*",B"P'P!G/23%*L!566"#!"()ZY5;6[,!+#!Y8N\Y8Y#!

5N#! aDC3121L!M#M#L!%*!/A#L!W+"'#.,.b20e['-'#"*+(4')(%'-&&"&&.,>'B/.,.B-/'&I.//&P!?22!e2*%C2!$%&L!5669#!!"#Z;[,!+#!87;\I"#!

5;#! W/C&%2L!P#$#L!%*!/A#L!K&&"&&#",*'()'B/.,.B-/'B(#@"*",B"'?&.,>'($F"B*.M"'&*%?B*?%"4'"1-#.,-*.(,P!'C!$%&!ML!"Y7N#!!ZNYNN[,!+#!887\N"#!

57#! ]1AA1/04L!P#(#L!a-M"'&*-,4-%4.R"4'@-*.",*'"1-#.,-*.(,&'&*((4'*+"'*"&*'()'*.#"'-,4'"1@"%.",B"d!O%/3)!G%/C2!$%&L!5668#!!&Z5[,!+#!5"N\55#!

5I#! 'DTA%*L!M#P#L!%*!/A#L!V&.,>'&*-,4-%4.R"4'@-*.",*&'*('-&&"&&'*+"'.,*"%@"%&(,-/'&I.//&'()'@+Z&.B.-,&P!?3/&!$%&L!"YYI#!)"Z"6!ET++A[,!+#!EY8\;#!

5Y#! $/CRDA14L!$#M#L!%*!/A#L!K,-/Z&.&'()'*+"'%"/-*.(,&+.@'$"*L"",'&B(%"'B(#@(,",*&'(,'-'&*-,4-%4.R"4'@-*.",*'B/.,.B-/'&I.//&'"1-#.,-*.(,P!?3/&!$%&L!5669#!)#Z"6!ET++A[,!+#!E;I\7"#!

96#! _1@L!?#L!%*!/A#L!A"&&(,&'/"-%,"4')%(#'&.1'Z"-%&'()'.,*"%,-*.(,-/'-4#.,.&*%-*.(,&'()'*+"'023!Nf&'=Eb$-&"4'B/.,.B-/'&I.//&'-&&"&&#",*P!?3/&!$%&L!"YYI#!)"Z"[,!+#!I8\Y"#!

9"#! 'C/1AD@4JQL!F#?#L!S#!(C/2&g$/14D2L!/2&!M#!G%43D+L!K'/-%>"b&B-/"'#?/*.B",*"%'($F"B*.M"'&*%?B*?%"4'B/.,.B-/'"1-#.,-*.(,')(%'/.B",&?%"P!?3/&!$%&L!"YY5#!&)Z"6!ET++A[,!+#!E97\Y#!

95#! O/0BAQ2L!P#L!%*!/A#L!K&&(B.-*.(,'$"*L"",'/.B",&.,>'"1-#.,-*.(,'&B(%"&'-,4'%"&(?%B"'?&"'-,4']?-/.*Z'()'B-%"'.,'@%.#-%Z'B-%"'@%-B*.B"P!M?$?L!"YYI#!$#'Z""[,!+#!YIY\Y;#!

99#! O/0BAQ2L!P#L!%*!/A#L!K&&(B.-*.(,'$"*L"",'/.B",&?%"'"1-#.,-*.(,'&B(%"&'-,4'@%-B*.B"'.,'@%.#-%Z'B-%"P!M?$?L!5665#!$##Z59[,!+#!96"Y\5;#!

98#! O/0BAQ2L!P#L!%*!/A#L!E+Z&.B.-,'&B(%"&'(,'-',-*.(,-/'B/.,.B-/'&I.//&'"1-#.,-*.(,'-&'@%"4.B*(%&'()'B(#@/-.,*&'*('#"4.B-/'%">?/-*(%Z'-?*+(%.*."&P!M?$?L!5667#!$*#ZY[,!+#!YY9\"66"#!

9N#! (C/2&g$/14D2L!S#L!%*!/A#L!A-%>"b&B-/"'?&"'()'-,'($F"B*.M"\'&*%?B*?%"4'B/.,.B-/'"1-#.,-*.(,')(%'/.B",&.,>')-#./Z'@+Z&.B.-,&P!F$?ML!"YY5#!!%&Z"6[,!+#!"79N\86#!

9;#! P%d213JL!P#L!%*!/A#L!K,'($F"B*.M"'&*%?B*?%"4'B/.,.B-/'"1-#.,-*.(,')(%'*+"'/.B",*.-*"['%"@(%*'()'*+"'@./(*'@%(F"B*'()'*+"'!"4.B-/'2(?,B./'()'2-,-4-P!?3/&!$%&L!"YY5#!&)ZI[,!+#!8I7\Y8#!

97#! hE$GXL!h#E#$#G#X#F#\#!V=!A0''V,.*"4'=*-*"&''!"4.B-/'A.B",&.,>'01-#.,-*.(,P!56"9!!56"9!$/C3)!5;=>!?@/1A/BA%!:CD0,!)**+,--...#T40A%#DCR-#!

9I#! V%!F)/0+A/12L!?#L!%*!/A#L!K&&"&&.,>'*+"'?,4"%/Z.,>'&*%?B*?%"'()'*+"'V,.*"4'=*-*"&'!"4.B-/'A.B",&.,>'01-#.,-*.(,'=*"@'6'*"&*'()'B/.,.B-/'&I.//&'?&.,>'B(,).%#-*(%Z')-B*(%'-,-/Z&.&P!?3/&!$%&L!566;#!#!Z"6!ET++A[,!+#!E"7\56#!

9Y#! UTL!a#L!%*!/A#L!K&&"&&.,>'B/.,.B-/'B(#@"*",B"['-'@./(*'@%(F"B*'*('"M-/?-*"'*+"')"-&.$./.*Z'()'-'&*-,4-%4.R"4'@-*.",*'bb'$-&"4'@%-B*.B-/'"1-#.,-*.(,'-&'-'B(#@(,",*'()'*+"'=L.&&'B"%*.).B-*.(,'@%(B"&&P!E.144!$%&!]JAQL!566;#!!"&Z5N\5;[,!+#!9Y5\Y#!

86#! '/CCD.4L!W#E#!/2&!E#!?BC/)/04D2L!W+"'E%(>%-##"4'E-*.",*['K'W"B+,.]?"')(%'K@@%-.&.,>'=*?4",*'E"%)(%#-,B"'.,'2/.,.B-/'c"?%(/(>ZP!$%&!X&T3L!"Y;8#!"*ZI[,!+#!I65\I6N#!

8"#! '/CCD.4L!W#E#L!=.#?/-*"4'@-*.",*&'.,'#"4.B-/'*"-B+.,>P!F/2!$%&!?44D3!ML!"Y;I#!*#Z"8[,!+#!;78\;7;#!

85#! FDAA124L!M#S#!/2&!P#$#!W/C&%2L!W+"'V&"'()'_"-/'E-*.",*&\'=.#?/-*"4'E-*.",*&'-,4'=.#?/-*(%&'.,'2/.,.B-/'01-#.,-*.(,&P'K!00'!"4.B-/'04?B-*.(,'N?.4"'c('8:P!$%&!O%/3)L!"YYI#!$'Z;[,!+#!N6I\5"#!

89#! WTBB/C&L!M#S#L!%*!/A#L!K,'($F"B*.M"'"M-/?-*.(,'()'B/.,.B-/'B(#@"*",B"P'''P!a!X2RA!M!$%&L!"Y;N#!$)$Z5N[,!+#!"95"\I#!

88#! FA%A/2&L!M#?#L!^#!?B%L!/2&!M#M#!P%*)/24L!W+"'?&"'()'&.#?/-*"4'@-*.",*&'.,'#"4.B-/'"4?B-*.(,['K!00'N?.4"'c('g6P!$%&!O%/3)L!566Y#!"!Z;[,!+#!877\I;#!

8N#! E3)2/B%AL!^#S#L!=.#?/-*.(,'-?&'3/".&B+'?,4'O/?*['=B+-?&@."/@-*.",*",L!12!=.#?/-*.(,'.,'4"%'!"4.R.,L!$#!E*#!S1%CC%!/2&!(#!'C%T%CL!X&1*DC4#!56"9#!+#!""N\"Y#!

8;#! $FFiX#!!"4.B-/'2(?,B./'()'2-,-4-'J?-/.)Z.,>'01-#.,-*.(,'5!22J0;#!!<31*%&!56"9!5;#69#56"9=>!?@/1A/BA%!:CD0,!)**+,--...#033#3/-%2-#!

87#! E3)T.1C*)L!G#]#L!%*!/A#L!X('&+(%*'B-&"&'"/.B.*'4.))"%",*'*+.,I.,>'@%(B"&&"&'*+-,')-B*?-/'I,(L/"4>"']?"&*.(,&'4(d!$%&!X&T3L!566"#!"(Z8[,!+#!98I\N;#!

8I#! e24*1*T*!:jC!?T4\L!]#T#K#2#e#:#$#G#!h(@"*",*'@%S)",'b'a-,4$?B+'R?%'E/-,?,>\'X?%B+)S+%?,>'?,4'K?&L"%*?,>'M(,'3-B+-%R*@%S)?,>",'O"%,'"YYY!!<31*%&!56"9!5;#69#56"9=>!?@/1A/BA%!:CD0,!)**+,--...#10A#T21B%#3)-&1%24*A%14*T2R-/44%440%2*c+CT%:T2R%2-#!

8Y#! WD&R%4L!'#!/2&!M#W#!$3eACDQL!K,-/Z*.B'>/($-/'U=20'%-*.,>&'-%"'&",&.*.M"'*('/"M"/'()'*%-.,.,>P!$%&!X&T3L!5669#!")Z""[,!+#!"6"5\;#!

N6#! E3)%::%CL!E#L!!"#$%$&'()*+%&,+-.&'#$)&'+/#01"#+2"3$)*4+5./26+b!"#$%#&'()*!#'$+)($!"#$%&'()*++%&,)-.,/0"(1*+20.0&30&(405,3,&6.%5,0"0&50"(,+(7-8+0&()9,&,6:8b!"#$%&'()*"+,"-./01*0+2345678!566YL!$%&1d12143)%!K/JTA*k*!F)/C1*lL!'%CA12L!V%T*43)A/2&#!

N"#! FCD2B/3)L!G#M#L!2(")).B.",*'-/@+-'-,4'*+"'.,*"%,-/'&*%?B*?%"'()'*"&*&P!S4Q3)D0%*C1J/L!"YN"Z";[,!+#!5Y7\998#!

N5#! ?2RD::L!]#W#L!=B-/"&\',(%#&'-,4'"]?.M-/",*'&B(%"&P'L!12!04?B-*.(,-/'!"-&?%"#",*L!P#G#!O)DC2&1J%L!X&1*DC#!"Y7"L!?0%C13/2!FDT231A!D2!X&T3/*1D2,!]/4)12R*D2#!+#!N6I\;66#!

N9#! WD:4*%%L!^#]#'#L!W+"'B-&"')(%'B(#@%(#.&"'.,'"4?B-*.(,-/'&"/"B*.(,'-,4'>%-4.,>L!12!U,'04?B-*.(,-/'W"&*.,>L!E#'#!?2&%C4D2!/2&!M#E#!W%A013JL!X&1*DC4#!"YI9L!MD44%Q\'/44,!E/2!KC/23143D#!+#!"6Y\57#!

N8#! ^C/0%CL!?#L!%*!/A#L!2(#@-%.&(,'()'-'%-*.(,-/'-,4'-,'"#@.%.B-/'&*-,4-%4'&"**.,>'@%(B"4?%"')(%'-,'U=20P'U$F"B*.M"'&*%?B*?%"4'B/.,.B-/'"1-#.,-*.(,&P!$%&!X&T3L!5669#!")Z5[,!+#!"95\Y#!

NN#! G/@%L!M#!/2&!X#!]%2R%CL!=.*?-*"4'A"-%,.,>P'A">.*.#-*"'@"%.@+"%-/'@-%*.B.@-*.(,#!"YY"L!F/0BC1&R%,!h21@%C41*Q!D:!F/0BC1&R%!SC%44#!

!