chief examiner’s report - biology · trial of practical assessment in leaving certificate...

Trial of practical assessment in Leaving Certificate Physics, Chemistry, Biology - 2017

Chief Examiner’s Report - Biology

This report is based on the marks assigned by the examiners in the trial, on the reports submitted by the examiners and advising examiners, and a review of the student work produced in the trial. 1. Introduction

1.1 Purpose of trial The State Examinations Commission (SEC), in consultation with the National Council for Curriculum and Assessment (NCCA), was asked by the Department of Education and Skills (DES) to carry out a trial of practical assessment in Leaving Certificate Biology, Chemistry, and Physics. The NCCA in its draft subject specifications (2013) for these subjects recommended that there should be a 90-minute externally assessed practical examination, worth 30% of the overall marks in each subject. Students would be assessed as they performed practical tasks and on a task booklet they completed during the practical session. As well as the practical assessment there would continue to be a written examination, worth 70%, in each of the sciences. The purpose of the trial was to assess the feasibility of including the proposed model of practical assessment as a component of these Leaving Certificate examinations in the future. The trial therefore was concerned with estimating roll-out costs and assessment of value versus cost, the validity of the practical tasks generated for the students, and whether such tasks could have ongoing validity. 1.2 Description of practical examination The trial involved students from Year 2 of Leaving Certificate and took place in October 2017. The practical examination in Biology was organised into sessions. The trial involved up to three 90-minute sessions in an examination centre (laboratory) in a day. Each session involved up to twelve students observed by one external examiner, each student working on a different task. The examiner awarded marks (up to 60) for the practical performance observed. The booklets completed by the students during the practical were collected and awarded marks (up to 60) later. 1.3 Number of students and the number of schools All 712 post-primary schools were invited to express interest in participating in the trial. Responses were received from over two hundred schools. The trial aimed to involve some schools in in just one of the subjects, Biology, Chemistry, Physics, while others would trial practical examination in two or all three subjects. Twenty-eight schools was the minimum required as per Figure 1.

1

Figure 1 – Schools in the Science Trial

Thirty of the applicant schools were selected to give a good geographical spread and a range of school types (size, gender of students, management type, language of instruction, DEIS status, etc.). The initial cohort of trial students in the schools selected was about 1600. The number of students that actually participated was about 1100. The fall-off is attributed to students opting out of the trial. Students aged 18+ could opt out by informing their subject teacher. Students (under age 18 at the time of the trial) could also opt out of participation with the permission of a parent or guardian provided in a letter to the school. The opt-out was to avoid any stressful impact on students in the trial preparing for their Leaving Certificate, given that involvement in the trial was not rewarded, and to ensure positive engagement of those students who participated in the trial. 2. The tasks

2.1 Structure of tasks Each student in a session was assigned a task at random. Each task was presented in a 4-page task booklet. For the Biology trial fourteen tasks were produced, twelve (designated B1, B2, B3 …..B12) that were used in the trial and two (B13 and B14) that were issued to trial schools as sample tasks to help students prepare for the trial. The level of difficulty and time required were intended to be the same for each task. The tasks were based on the mandatory practical activities in the current Biology syllabus. Similarly, fourteen Chemistry and fourteen Physics task booklets were also produced for the Chemistry and Physics trials. A common separate Instructions and information for the practical examination sheet was issued to each student on the day and a common format/layout was used for the booklets in each subject and across the three subjects, although the content of the tasks in each subject was distinct to that subject. All tasks were presented at common level. In Biology, nine of the twelve trial tasks consisted of a single activity, while three were divided into two sub-tasks each. Each task and sub-task had a title stating the nature of the investigation involved, a short introductory paragraph, followed by a list of instructions, followed again by a set of questions with spaces for answers. The external examiner evaluated the individual practical performances as the students carried out the tasks, each student working alone. Students could ask for help if they needed it but the examiners did not question the students directly. Pseudodata were available for students who could not,

2

for whatever reason, generate their own data. Certain types of assistance or the issue of pseudodata may have been penalised by the examiner. The task booklets were used by the students to record observations, data, and data analysis and to answer questions either during or after their experimental work or a combination of both. Graphs were drawn on standard SEC graph paper. Tidying up the work stations at the end of the practical examination session was part of each task. At the end of the session the task booklets and any graph paper issued were collected by the examiner to be marked later. For students whose normal mode of working in the laboratory involves using a PC or tablet, and who, in consultation with the SEC with regard to Junior Certificate examinations, are likely to be granted reasonable accommodations with respect to the use of a computer for written Leaving Certificate examinations in June 2018, pdf-fillable versions of some tasks in each subject were prepared so that students could access the trial in that way. No Biology student in the trial used these digital versions of the tasks. 2.2 Direct and indirect assessment Direct assessment rewarded practical skills that are difficult or impossible to test in a written examination, e.g.

• selecting, manipulating and/or assembling suitable apparatus, • taking accurate measurements, • tabulating, graphing and performing other analysis of own data, • working safely and efficiently in a laboratory and using resources economically, • following procedures or devising own method to carry out an investigation, etc.

Direct assessment was carried out by an examiner observing students as they carried out the tasks. Indirect assessment gave credit for the written work in the task booklets and accompanying graph paper. Indirect assessment was carried out by examiners marking the task booklets produced in the practical session, and was carried out between two and three weeks after the direct assessment. 3. The trial practical examination 3.1 Description of what happened during the examination Students were invited to enter the examination centre (laboratory) about ten minutes before the session start time. Students had been randomly assigned by the examiner to a certain task on a Direct Assessment Mark Sheet, TPA-6. As students entered they were directed to the corresponding numbered workstation. They were given a sticky label printed with their Trial ID number to wear during the practical session. Students were reminded to wear appropriate safety clothing and that that mobile phones were not permitted in the examination centre. Students were asked to read the ‘Instructions and information’ sheet on their workstations. Students were reminded that they were allowed to ask for help, by putting up their hand, and, depending on the help needed, that such help might involve losing some marks. The examiner distributed the task booklets, each student getting a different task (or with the minimum possible degree of duplication). The students were reminded that the first ten minutes of the examination session was for reading the task booklet, preparation and

3

planning. Students were given permission to start and were informed when the first ten minutes was up and notified that they would be alerted again ten minutes from the end and that the last five minutes would be for clearing up. During the first ten minutes students read through their task and collected apparatus, chemicals and other materials. They were not permitted to assemble the apparatus or start work on the task or on completing the booklet during that time. During the next 75 minutes students assembled the selected equipment and manipulated resources, carried out practical activities, and recorded observations and measurements. The examiner used a Direct Assessment Mark Sheet on a clipboard to assign marks/make notes discreetly as he/she moved around the laboratory. The examiner attempted to make a fair assessment of each individual student’s practical abilities in five categories according to the assessment objectives, and recorded their assessments on the mark-sheet. The examiner attempted to give equal attention to all students. There should have been no communication between the examiner and a student other than exchanges about a request for help by a student or a student calling the examiner to examine a particular activity. The examiners did however intervene in situations that threatened the safety of the student or of other students or compromised the efficient running of the examination centre. Where a student needed help that involved a penalty, the help required was given to enable the student to progress. The penalty was recorded in the appropriate place on the mark sheet. As the session progressed the examiner had ample opportunity to observe the students’ selection of apparatus, chemicals and other materials; assembly of apparatus; use of apparatus; and safe and efficient way of working in the laboratory. Towards the end of the session the examiner awarded marks for each of these four categories to each student based on the student’s overall 90-minute performance. The lowest mark in each category was 0; there was no negative marking in any category even when penalties were applied for errors or help given. For some specific tasks, there were relatively few opportunities for the examiner to observe a student making a measurement or recording an observation. The examiner may have asked students doing these tasks to call him/her to observe key moments or, if appropriate, e.g. if time permitted, the examiner may have asked students to repeat a key step involving measurement/observation. Marks for this category were awarded to each student based on the student’s overall 90-minute performance and were deducted where help was given or data provided. The lowest possible mark in this category, even when penalties applied, was also 0. Ten minutes before the end of the session the students were reminded of the time and that the last five minutes was designated for tidy-up. An announcement was made when just five minutes was left that students should stop working on the tasks and start cleaning and tidying their workstation.

4

At the end of the examination session the examiner instructed students to leave their task booklets, with any graph paper used inserted inside the booklet, at their workstations. The examiner completed his/her session mark-sheet and collected the session booklets which were sent to SEC Athlone by registered post. 4. Assessment objectives of practical examination

4.1 Direct assessment

4.1.1 Marking The 60 marks available for direct assessment were awarded by the external examiner who supervised the practical session in five categories, as follows. 1. Selection of apparatus, chemicals & other materials

- apparatus suitable for task - sufficient apparatus appropriate to task - chemicals/other materials needed for task

2. Assembly of apparatus - correct assembly - manipulative skills in assembly

3. Use of apparatus - candidate carries out task as directed - manipulation of apparatus during conduct of task - co-ordination and dexterity in the use of equipment - apparatus used appropriately

4. Observations/measurements - correct observation/measurement technique - accurate observations/measurements - sufficient repetition where appropriate

5. Working safely & efficiently & cleaning up - personal safety and safety of others - safe and economic use of resources - tidy work practices - task carried out in the correct sequence - task completed within the given time - adherence to safe work practices in relation to electrical appliances, glassware, hot

liquids, chemicals, spillages, etc. - cleaning work area.

5

Each assessment objective carried up to 12 marks, and the marks that could be assigned by the examiner for each objective were 12, 8, 4 or 0 according to the marking key given in Table 1 below.

a. Indirect assessment

Table 1 – Direct Assessment Marking Key

4.1.2 Moderation Where indicated, moderation of marks according to an advising examiner’s instructions was applied to the initial marks assigned by the examiners. Moderation was based on a comparison of the marks applied by an advising examiner and those applied by an examiner marking a session in parallel without conferring. Where the examiner’s marks differed from the adviser’s to a degree that exceeded a previously set tolerance, an adjustment, indicated by the adviser, was applied to all the marks of the examiner.

4.2 Indirect assessment The 60 marks available for indirect assessment were awarded to the work in a student task booklet by an external examiner Each examiner applied a marking scheme for the tasks that had been discussed and agreed at a marking conference. The work of each examiner was monitored by an advising examiner during the marking.

5. Practical examination outcomes – direct assessment 5.1 Statistics 5.1.1 Average overall total score for direct assessment Five hundred and twenty-three (523) students participated in the Biology trial. Nineteen (19) were examined through the medium of Irish. Because of varying patterns of course coverage in the trial schools some students were unfamiliar with certain tasks and in such cases other tasks were substituted. The number of times each Biology task was trialled, and the average direct assessment mark for each task is given in Table 2.

DIRECT ASSESSMENT MARKING KEY

High level of achievement 12 Low level of achievement 4

Moderate level of achievement 8 Not achieved 0

6

Task

No

students (and as

%)

Rank order

no students

Average DA mark out of 60 (and as

%)

Rank order

average DA mark

Task Topic(s)

1 24 (4.6) 11 52.9

(88.2) 1 Photosynthesis rate

vs CO2 concentration

2 38 (7.3) 8 48.4


vs light intensity

3 14 (2.7) 12 48.6

(81.0) = 5 Effect of IAA on plant tissue

4 68 (13.0) 1 47.6

(79.3) 8

o Examining plant cells with a microscope

o Examining animal cells with a microscope

5 58 (11.1) 3 43.7

(72.8) 12 o Qualitative food tests o Enzyme denaturation

6 56 (10.7) 4 45.4

(75.7) 10 Enzyme activity vs pH

7 53 (10.1) 5 49.1

(81.8) = 3 Enzyme activity vs temperature

8 26 (5.0) 10 45.2

(75.3) 11 Plant tissue DNA isolation

9 65 (12.4) 2 47.1

(78.5) 9

Effect of solute concentration on water movement across a

selectively-permeable membrane

10 47 (9.0) 6 48.6

(81.0) = 5 Mammalian heart dissection

11 28 (5.4) 9 49.1

(81.8) = 3

o Animal population quantitative survey

o Seed germination conditions

12 46 (8.8) 7 49.5

(82.5) 2 Growing leaf yeast

Table 2 – Number of times each Biology task was trialled, and average direct assessment mark for each task

The overall average direct assessment mark in the Biology tasks was 48.1 out of 60 or 80.2% (with a standard deviation of 9.7 marks or 16.2%) and a mark range [4, 60]. Graph 1, below, shows the distribution of direct assessment marks (expressed as a percentage).

7

Graph 1 – Direct Assessment Mark Distribution (%) – Biology

Graph 2 shows good correlation between examiners’ original marks and the moderated marks.

Graph 2 – Comparison of examiners’ marks and marks following moderation – Biology

-20

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

No

of S

tude

nts

Direct Assessment Mark (%)

y = 0.9668x + 1.8099R² = 0.9635

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Exam

iner

Mar

k

Moderated Mark

Comparison of Examiner Mark and Moderated Mark

8

5.1.2 Average score for each sub-part of direct assessment Table 3 shows the average mark assigned by examiners in each of the direct assessment objectives. These results show that most students were able to select suitable apparatus, chemicals and other materials and assemble the equipment, even by trial and error, to carry out their tasks. Some errors, inaccuracies, poor technique and safety issues were observed by examiners as students used the apparatus and took measurements. Most students worked safely and efficiently. The vast majority left their work stations tidy.

Assessment Objective Average

mark out of 12

Average mark

% Selection of apparatus,

chemicals & other materials

10.7 89.2

Assembly of apparatus 9.8 81.7

Use of apparatus 8.6 71.7

Observations / measurements 8.7 72.5

Working safely & efficiently & cleaning up 10.1 84.2

Table 3 – Average scoring in direct assessment objectives

5.1.3 Commentary on the statistics Examiner reports recorded that students in the trial engaged very enthusiastically and direct assessment results show that most were able to work competently and safely in the laboratory and complete their tasks. The tasks were written for common level so, given that approximately three quarters of Biology candidates are expected to sit the 2018 Leaving Certificate examination at Higher level, it is not surprising that most students were able to exhibit a high level of achievement in direct assessment. More than 37% of the students achieved 90% or more of the available direct assessment marks; 9.9% achieved full marks. This was in line with marking criteria where full marks were awarded for reaching a high (and not necessarily flawless) level of achievement in each objective. The range of average direct assessment marks across the Biology tasks used in the trial was [43.7, 52.9]. This relatively narrow range suggests that the tasks were reasonably similar in level of practical difficulty.

9

5.2 Comments on direct examining based on examiners’ reports 5.2.1 Student performance Selection of apparatus, chemicals & other materials Students generally did very well in selecting the correct apparatus, chemicals and materials for their tasks and most scored highly in this area of assessment. However, some students in some schools were not familiar with the names of some items in the equipment list for their task e.g. boiling tubes, mass balance, cork borer, catalase source. A small minority in some schools could not identify glass slides and coverslips. In some cases students collected beakers where test tubes were the correct choice, or graduated cylinders where beakers were intended, and this sometimes led to a shortage of beakers or graduated cylinders for those students whose tasks actually required them. Sometimes such misunderstandings were remedied by trial and error or by watching other students, but sometimes they were not, which led to problems later in the tasks. In general chemicals were supplied in clearly labelled containers and this allowed for easy recognition and collection. However, in cases where beakers had been chosen rather than more appropriate test tubes, inappropriately large aliquots of particular chemicals were collected in these beakers, leading to shortages elsewhere. To a large extent, students who had either made a mistaken choice, or had not collected an item they later discovered they needed, revisited the apparatus repository and rectified the situation. This was allowed for in the marking protocol, and was commented upon favourably by many examiners. In most schools the equipment to be used was made available by the designated support teacher (DST) in a single area in the laboratory being used as the examination centre. The equipment was most often made available on a trolley or set of trolleys, or sometimes on a bench at the back or front or side of the lab. There were no reports of tasks being impossible to do because of lack of equipment or chemicals although some schools were short of some items of equipment such as thermostatically controlled water baths and accurate mass balances. In some laboratories, students had to queue to use an accurate balance. This could easily be addressed if schools acquired a number of modern, cheap, jeweller’s balances. It was commented upon by many examiners that a full implementation of practical assessment in Biology, along the lines of the trial, would provide an impetus for a radical re-evaluation and restructuring, by both teachers and students, of the approach to teaching and learning biology as a practical science subject. Assembly of apparatus and preparation of biological samples Despite there being a clear disparity in ability levels among students, both between schools and within schools, examiners generally reported that the assembly of apparatus and preparation of biological samples were well performed. This phase was more straightforward for some tasks than for others. For instance, the heart dissection in task B10 and the leaf yeast investigation in B12 were easier to set up than the photosynthesis tasks B1 and B2, or the enzyme investigations, B5.2, B6 and B7. However, the same few difficulties experienced by students were mentioned in many examiners’ reports; these were:

10

• In centres where thermostatically controlled waterbaths were not available many students demonstrated unfamiliarity with the assembly and preparation of alternatives.

• Lighting a Bunsen burner and regulating the flame proved problematical for some students, to the extent of unsafe practice in some cases.

• Many students did not label solutions once they had collected them, which had the potential to cause confusion where more than one solution was being used in a given task.

• Some students did not add water to the pieces of celery before blending when preparing to extract catalase in tasks B5.2, B6 and B7. If a blender was not used, many students did not chop the celery into small enough pieces, some using inappropriately large chunks. Similarly, some students did not seem to be familiar with how to find and remove the epidermis from onion leaves, some attempting to examine quite large pieces of the fleshy leaves with the microscope.

• Elodea stalks were often cut in air before immersion in NaHCO3 solution. • Most students seemed unfamiliar with cork borers for cutting discs from the leaves

in the leaf yeast task.

Use of apparatus Again, notwithstanding the wide range of ability encountered, most students had a reasonable idea how the equipment should be handled and generally did so safely; there were few spillages or breakages. Almost all worked more quickly than they needed to, however, which often led to examiners missing crucial steps in certain tasks. Tasks with more comprehensive instructions were generally better performed although sometimes the detailed instructions were not followed, either through being ignored, not being read properly, students not understanding certain terminology, or through rushing. In tasks B5.1 (food tests) and B11.2 (germination conditions), and to a lesser extent in task B9 (osmosis) itemised procedural instructions had not been included to the same extent as in the other tasks. This was because these three practical activities should have been very familiar to the vast majority of the trial students, being part of the Junior Certificate Science syllabus they would have studied, and it was considered a valid avenue of enquiry to test the extent to which this would make a difference to student performance. Examiners duly reported widespread difficulty among students because of the lack of detailed instructions in these three tasks/ sub-tasks. Overall, examiners reported apparent unfamiliarity among some students with the correct procedure for focusing a microscope; with the use of dividers when measuring the thickness of the heart chamber walls; with the importance of and adherence to aseptic technique; with the necessity of heating Benedict’s solution when testing for reducing sugars; with the necessity to use a new dropper or syringe with each new solution; with methods of controlling the temperature of non-thermostatically controlled waterbaths. Interestingly in this regard, in one school a brand new thermostatically controlled waterbath, obtained specifically for the trial, was ignored by students who elected to use the Bunsen/ beaker/ thermometer method; and in another school students doing task B8 ignored a thermostatically controlled waterbath, set up at the task B8 workstation, that had been set at 60 °C by the DST was similarly ignored. Both sets of students were significantly delayed in their work by this choice. Examiners also reported that students had difficulty manipulating

11

the Visking tubing in task B9, measuring volumes accurately, and measuring the pH of solutions, although it could have been that students thought the latter procedure unnecessary if the supplied buffer solutions were clearly labelled with their pH value. A problem outside the students’ control was that in many centres the hydrogen peroxide supplied for tasks B5.2 , B6 and B7 was either supplied at an incorrect concentration or was of such poor quality otherwise as to have been effectively useless for its intended purpose. The most widespread problem outside the students’ control was the poor condition of the pondweed Elodea canadiensis that was needed for the two photosynthesis tasks B1 and B2. In some centres examiners reported that the DST had not made any effort to prepare the Elodea by illuminating it for a period before the start of the task sessions. Some DSTs supplied Cabomba spp. instead, which seemed to work better but Cabomba is an invasive plant the use of which is discouraged for that reason. Many examiners reported students feeling under pressure because they were performing their task alone rather than in the pair or group scenarios more familiar to them from classroom experience. At present, the success of mandatory practical activities in school is generally considered to be the responsibility of the teacher and not of the individual student. When experiments fail to deliver the expected product or result, the student is not usually required to troubleshoot the situation themselves and is not often required to care about the outcome and may even, if queried, attribute blame to the equipment, to a chemical, to his or her partner or to the teacher. If practical assessment, as trialled, were introduced, students would realise that in laboratory lessons they need to accept responsibility for the conduct and outcomes of their own experiments as they practice for external practical assessment. However, their tolerance for inadequate equipment and chemicals such as ‘hydrogen peroxide’ that had actually previously reacted would also change. Observations / measurements Most students understood what was to be observed or measured and knew the required technique but the execution was not always competent. Observations and measurements in the enzyme tasks B6 and B7 were reported as having been well done overall, although the use of pH meters was rare, and not of a very high standard when employed. Generally, use of the electronic mass balances was good. Most students measured volume accurately, however, a significant minority did not make sure the graduated vessel was on an even surface before attempting to measure volume at the meniscus. Some did not know to read volume at the meniscus, and sometimes careless reading of instructions led to incorrect volumes being measured. Volume measuring with Pasteur pipettes was reported as poor, with students failing to ensure the absence of bubbles from the liquid in the pipette. Students were occasionally observed removing the thermometer from the solution in order to measure the temperature of the solution. Examiners reported that students were frequently careless about making sure that the time intervals between successive measurements were equal, as specified in the procedure. Pseudodata were often needed for tasks B1 and B2, the photosynthesis investigations, because of the non-performance of the pondweed. Where pseudodata were supplied good graphs were generally drawn from them. Many examiners reported students making up results, based on past experience or on theory, rather than ask for the pseudodata, even

12

though they would not have been penalised under the circumstances. Such fabrication of results was also sometimes reported in Tasks B6 and B7. There was some evidence of poor calculations in completing the tables in B6 and B7. For instance, some candidates proved unable to arrive at a correct figure for ‘volume of foam produced’ by subtracting the initial volume from the final volume. Similarly, in task B1, there was some evidence of inability to calculate the average number of bubbles produced from the three attempts at each CO2 concentration. Examiners also occasionally encountered inaccurate drawings of observed plant and animal cells in task B4, poor descriptions of the turgidity/flaccidity of the Visking tubing in task B9, poor flag labelling of the dissected heart and poor measuring of the thickness of the chamber walls, both in task B10, and confusion in applying the instructions and formula for the capture-recapture simulation in task B11.1. Most surprising was the lack of ability in conducting and generating results from the qualitative food tests in task B5.1, given that this practical is on the Junior Certificate syllabus. Examiners encountered very few instances of students repeating a task in order to generate confirmatory data. In certain tasks (B1, B2, B6, B7, B8, and especially B5) this was at least partly explained by the time consuming nature of the task. Some examiners and advising examiners recommended that tasks should directly examine students’ proficiency at performing certain generic skills such as measuring liquid volumes, filling pipettes, using a balance, preparing a microscope slide for examination, separately from the context of a particular investigation. Some examiners and advising examiners decried what they saw as the overly prescriptive nature of the instructions in the tasks and advocated the assessment of students’ ability to come up with practical solutions to biological problems based on the skills they had learned during their laboratory work for the Leaving Certificate course. It may be that such an approach to practical assessment in Biology would be more realistic once the new specification is implemented. All advising examiners were of the opinion that it would have been very useful if examiners had been allowed to ask questions of the students as they observed their work. Working safely & efficiently & cleaning up Most examiners reported that students worked safely in a systematic and efficient manner. Few spillages or breakages were reported and those that did occur were quickly cleaned up by the student. White coats were available in most schools, and where available were always worn. A minority of schools supplied aprons instead of white coats. The aprons were thought by examiners to be unsatisfactory as they were generally plastic coated and too short. Chemicals were generally used economically although in some cases if a student used an inappropriately large container to collect a particular chemical e.g. a beaker instead of a test tube, the volume of chemical collected was consequently too large. This practice led to shortages of some chemicals in some schools. Most students worked systematically through the instructions in the procedure section of their task, with those tasks having the most comprehensive instructions being most efficiently carried out. A minority of students, however, seemed to have largely ignored the instructions and carried out the task as they remembered having done it in lab class. Students generally moved carefully around the lab and for the most part worked quietly and efficiently.

13

The most commonly reported unsafe practice was students not wearing safety goggles or latex gloves when handling the hydrogen peroxide in tasks B5.2, B6 and B7; and when handling the IAA solutions in task B3. Other examiners reported long hair not being tied back, some students not washing the heart before dissecting; having electric leads too close to hotplates; overfilling Bunsen burner-heated waterbaths; using the scalpel unsafely. The majority of students kept their workstations clean, and tidied up as they went along. The messiest practice observed was students cutting plant material on the lab bench surface instead of using a chopping board. The final tidy-up was done to a high standard with almost all workstations being left in good order for the next student. Just one school was reported in which the students were not good at cleaning and tidying up after themselves. The Biology tasks were not all of equal demand in terms of time needed, with some students finishing up to 30 minutes earlier than those who were assigned the more time consuming tasks. Task B5 was reported by all advising examiners as being the most difficult for students to finish on time. If practical assessment, as trialled, were introduced it would prompt discussion about best, safe laboratory practice. A detailed set of protocols could be issued by the SEC to schools to alert schools, teachers, students and examiners to a common, acceptable safety code in respect of safety clothing, labelling, fume-hood use, etc. 5.2.2 Issues that affected student performance in direct assessment In general the issue that most impinged upon students’ ability to perform the Biology tasks was unfamiliarity with either the task they had been given or with certain terms in the task booklet, such as names of items of apparatus or particular terms or phrases in the instructions. Taking into account the fact that the trial was conducted at a common level, and that students of widely divergent ability were assessed, it was nevertheless clear from examiners’ and advising examiners’ reports that in some schools practical laboratory work is done in a way that allows students to engage with the process and to take control of their own learning to a significant extent, while there are other schools in which this is not the norm, or at least not to the same extent.

There were some particular issues that affected student performance in the individual tasks.

Task B1 (Photosynthesis rate vs CO2 concentration) The main issue here was the poor quality of the supplied Elodea pondweed. In the great majority of cases students who attempted to perform this task had to ask for pseudodata. Task B2 (Photosynthesis rate vs light intensity) Here again the main issue was the poor quality of the supplied Elodea pondweed. Similarly to B1, in the great majority of cases students who attempted to perform this task had to ask for pseudodata.

14

Task B3 (Effect of IAA on plant tissue) In many schools this practical had not yet been covered and students were assigned an alternative task. Where it was done there were no particular issues that affected student performance. Task B4 (Examining plant cells and animal cells with a microscope) Examiners reported that the × 40 objective lenses were often of inferior quality and did not give good quality images. The cause of the this difficulty may have been cheap lenses, but it was reported by examiners that some students’ poor skills at microscope manipulation led to frequent observations of × 40 objectives being so carelessly used that they were driven through the coverslip/ slide on the stage. Such abuse would inevitably damage the optical quality of these lenses. Task B5 (Qualitative food tests; Enzyme denaturation) Detailed instructions were not given for the qualitative food tests in sub-task 5.1. These food tests are part of the Science syllabus that the vast majority of trial students would have studied for their Junior Certificate. It was considered a valid avenue of enquiry to test the extent to which that fact would make a difference to student performance. Examiners reported widespread difficulty among students because of the lack of detailed instructions in this sub-task. Biuret reagent was not given in the equipment list for sub-task 5.1. Instead the constituents of biuret solution (CuSO4 and NaOH) were listed and this led to confusion, as most school labs now use already formulated biuret compound. In sub-task 5.2 the procedure was written for doing the investigation using catalase. In a small number of schools students had done this investigation using amylase and were not familiar with the details of the catalase procedure. Also, while celery and radish were the catalase sources given in the DSTs’ equipment list, some students were used to using liver for this purpose, which, for various reasons, was unsuitable in the context of the trial. Also in sub-task 5.2, the hydrogen peroxide was either supplied at the wrong concentration (20% was specified in the equipment list but the DST could have mistaken this for 20 vol., which is equivalent to 6%) or was old or had been inappropriately stored and had already reacted and lost its potency. Task B6 (Enzyme activity vs pH) The catalase/ amylase and the hydrogen peroxide issues as mentioned above applied here also. pH meters supplied for use in this task were almost always uncalibrated, and students were not familiar with their operation in most cases. Task B7 (Enzyme activity vs temperature) The catalase/ amylase and the hydrogen peroxide issues as mentioned above applied here also. Task B8 (Plant tissue DNA isolation) In some schools this practical had not yet been covered and students were assigned an alternative task. Where it was done just one issue was reported: Even though the ice included on the DSTs’ equipment list was specifically for this task, in many schools it was,

15

understandably, not made available on the equipment table/ trolley. Consequently, and because it was not mentioned in the task equipment list, many students did not remember to use it at the appropriate stage of this task, electing instead to attempt to cool the beaker in tap water or to just leave it standing on the bench until the recommended time had elapsed. Either of those options, but especially the latter, would have had significant negative influence on the chance of isolating any DNA. Task B9 (Effect of solute concentration on water movement across a selectively- permeable membrane) No issues were reported that affected student performance in this task. Task B10 (Mammalian heart dissection) The main issue that arose for this task was that many students, on learning they had been assigned the heart dissection, elected not to take it, either through squeamishness or conscientious objection. Task B11 (Animal population quantitative survey; Seed germination conditions) For the same reason as applied to sub-task 5.1, detailed instructions were not given for the investigation into optimal seed germination conditions in sub-task 11.2. A similar outcome was reported. Task B12 (Growing leaf yeast) In most schools this practical had not yet been covered and students were assigned an alternative task. Where it was done no particular issue was reported as having affected student performance.

16

6. Practical examination outcomes – indirect assessment

6.1 Statistics

6.1.1 Average mark overall for indirect assessment The overall average indirect assessment mark in the Biology trial was 27.5 or 45.7% (with a standard deviation of 14.0 marks or 23.3%) and a range [0, 57]. The average mark value, the large standard deviation and the large range are indicators that the task booklets were discriminating of different levels of student ability. Graph 3 shows the distribution of the indirect assessment marks, expressed as percentages.

Graph 3 - Indirect Assessment Mark Distribution (%) – Biology

6.1.2 Average mark for each task Table 4 summaries the 523 student performances in indirect assessment in the Biology trial. Seventeen students submitted task booklets completed in Irish. Only 2.7% of the students achieved 90% or greater of the available mark, meaning that this part of the assessment was significantly more discriminating than the direct assessment part. The range of the average marks per task was [18.5, 42.4]. This range is broader than would be acceptable in a full roll-out situation, however, in this small sample, statistics can be very strongly influenced by the performance of just a few individuals and fluctuations in results from task to task are understandable. On full roll-out, it would be expected that the numbers taking each task would be similar and the results could be scaled to ensure that the same average mark was achieved in each task thus addressing issues related to reliability associated with different candidates completing different tasks.

0

10

20

30

40

50

60

70

80

90

0 10 20 30 40 50 60 70 80 90 100

No

of S

tude

nts

Indirect Assessment Mark (%)

Indirect Assessment Mark (%) Distribution

17

Task

No

students (and as %)

Rank order no

students

Average IA mark out of

60 (and as %)

Rank order by average

IA mark

Task Topic(s)

1 24 (4.6) 11 21.6


vs CO2 concentration

2 38 (7.3) 8 20.8


vs light intensity

3 14 (2.7) 12 31.0

(51.7) 4 Effect of IAA on plant tissue

4 68 (13.0) 1 42.4

(70.7) 1



5 58 (11.1) 3 26.8

(44.7) 5 o Qualitative food tests o Enzyme denaturation

6 56 (10.7) 4 18.5

(30.8) 12 Enzyme activity vs pH

7 53 (10.1) 5 22.6

(37.7) 8 Enzyme activity vs temperature

8 26 (5.0) 10 26.7

(44.5) 6 Plant tissue DNA isolation

9 65 (12.4) 2 23.7

(39.5) 7



10 47 (9.0) 6 34.3

(57.2) 3 Mammalian heart dissection

11 28 (5.4) 9 39.8

(66.3) 2



12 46 (8.8) 7 21.4

(35.7) 10 Growing leaf yeast

Table 4 ̶ Number of times each Biology task was trialled, and average indirect assessment mark for each task

6.1.3 Commentary on the statistics There was a wide variation in the quality of student answering in the Biology tasks, with marks for the indirect assessment varying from 0 to 57 out of 60. Examiners marked either 32 or 33 booklets each, with one examiner marking 34, so the number of any particular task booklet marked by each examiner was in the small single figures.

18

Given the small numbers of booklets involved, examiner ‘general’ impressions were sometimes contradictory e.g. almost every task was described by some as having been well done, while others said they were average or even poor. Probably reflecting the range of student ability, and the fact that the trial was at common level, questions seeking reasons behind particular steps in the various procedures were quite well answered by some students and considerably less well by others. Similarly, examiners reported many students having difficulty with such terms as reliability, significance, uncertainty, limiting factor, and variables, all of which figured across the indirect assessment of the tasks. Indeed the crucial concept of the variable, which was explicitly interrogated in five of the twelve Biology tasks and implicitly in some others, was very poorly understood by most students. Examiners also reported that a significant minority of students seemed to be unused to drawing graphs of their results, making many elementary mistakes in both labelling and scaling of the axes. Similarly, analysis of the graphs generated from their own results or from pseudodata was done poorly by many students. Students generally achieved higher marks where questions on safety appeared in the indirect assessment, although the major exception to this trend seems to have been an almost universal ignorance of the proper method of disposal of dissected hearts. 7. Direct and indirect assessment – contrast and comparison 7.1 Analysis of overall student performance The number of students who participated in the Biology trial was 523. The average total practical assessment mark per task was 75.6 or 63.0% (with a standard deviation of 19.45 marks or 16.21%) and a mark range [4, 118]. Table 5 shows the average total mark in each Biology task used in the trial. The range of average marks (out of 120) was [63.9, 90.0]. This range is broader than would be acceptable in a full roll-out situation, suggesting that the tasks were not of equal difficulty. However the total number of students involved in the trial was small, and the statistics were possibly influenced by the different states of readiness of the students for assessment given the time of the trial and fluctuations in results from task to task are understandable. On full roll-out, it would be expected that the numbers taking each task would be similar and somewhere between 2800 and 2900 per task. The results could be scaled to ensure that the same average mark was achieved in each task thus addressing issues related to reliability associated with different candidates doing different tasks.

19

Task

Average DA mark out of 60

(and as %)

Average IA mark out

of 60 (and as %)

Average total mark out of 120 (and as %)

Rank order by average total mark

Task Topic(s)

1 52.9 (88.2)

21.6 (36.0)

74.5 (62.1) 5

Photosynthesis rate vs

CO2 concentration

2 48.4 (80.7)

20.8 (34.7)

69.2 (57.7) 11

Photosynthesis rate vs

light intensity

3 48.6 (81.0)

31.0 (51.7)

79.6 (66.3) 4 Effect of IAA on plant tissue

4 47.6 (79.3)

42.4 (70.7)

90.0 (75.0) 1



5 43.7 (72.8)

26.8 (44.7)

70.5 (58.8) 10 o Qualitative food tests

o Enzyme denaturation

6 45.4 (75.7)

18.5 (30.8)

63.9 (53.3) 12 Enzyme activity vs pH

7 49.1 (81.8)

22.6 (37.7)

71.7 (59.8) 7 Enzyme activity vs

temperature

8 45.2 (75.3)

26.7 (44.5)

71.9 (59.9) 6 Plant tissue DNA isolation

9 47.1 (78.5)

23.7 (39.5)

70.8 (59.0) 9



10 48.6 (81.0)

34.3 (57.2)

82.9 (69.1) 3 Mammalian heart dissection

11 49.1 (81.8)

39.8 (66.3)

88.9 (74.1) 2



12 49.5 (82.5)

21.4 (35.7)

70.9 (59.1) 8 Growing leaf yeast

Table 5 - Average total mark for each task, Biology About 2.3% of the students in the Biology trial achieved 90% or greater of the available total mark; 15.7% were awarded 80% or greater. For comparison, the Grade 1 rate at Higher level Leaving Certificate Biology in 2017 was 4.9% and the combined Grade 1 and Grade 2 rates was 21.2%.

20

Below, graph 4 shows how the distributions of marks for direct and indirect assessment compare. Graph 5 shows how the sum of the direct and indirect assessment mark distribution compares with the distributions of marks for direct and indirect assessment. Graph 6 compares the direct assessment marks and the indirect assessment marks. The scatter plot in graph 6 shows that two different skill sets were examined. However, the scores are not fully independent of each other as the task booklets were completed at the same time as the direct assessment. For example, the results show that the 52 students who were awarded full marks in direct assessment were awarded marks in indirect assessment in the range [9, 57]. Some of these students may have worked through their tasks very well but at a rate that allowed too little time to complete the task booklets resulting in low indirect assessment scores. Other students may have had to repeat parts of their task to succeed in completing their tasks, could have been awarded a high direct assessment mark but at the expense of their indirect assessment mark.

Graph 4 – Comparison of mark distributions for direct assessment and indirect assessment – Biology

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

No

of S

tude

nts

Mark (%)

Comparison of Direct and Indirect Mark Distributions (%)

Indirect Assessment Direct Assessment

21

Graph 5 – Comparison of mark distribution for the sum of the direct and indirect assessment marks with the of mark distributions for direct assessment and indirect assessment – Biology

Graph 6 – Comparison of indirect assessment marks and direct assessment marks – Biology

0

20

40

60

80

100

120

0 10 20 30 40 50 60 70 80 90 100

No

of S

tude

nts

Marks (%)

Comparison of Direct, Indirect and Total Marks (%)

Indirect Assessment+IA!$K$580 Direct Assessment Total Mark

y = 0.2517x + 43.22R² = 0.1326

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Dire

ct A

sses

smen

t Mar

k

Indirect Assment Mark

Indirect and Direct Assessment

22

7.2 Attainment of Key Syllabus Objectives, Engagement and Performance

7.2.1 Psychomotor Domain The science trial provided new opportunities to assess student achievement in the psychomotor domain in terms of speed and precision of action, techniques acquired and to assess achievement of the practical the skills and competences specified in the syllabus. It is not possible to assess these skills by means of a written examination. Perception and Readiness to Act Each task provided opportunities to directly assess students’ ability to use their senses to inform actions, e.g. task B4, (Repeat the observation under medium and high magnification …. Draw and label an image of what you observed under high magnification) and, task B5.1, (After 2 minutes, add the hydrogen peroxide into the graduated cylinder containing the other reagents and immediately record the volume in the table below) also, task B8, (Carefully add 10 cm3 of ethanol to the filtrate. Record the temperature of the ethanol here….). Each task allowed students to demonstrate that they had achieved a readiness to act in the biology laboratory, i.e. an ability to prepare for, and then proceed through a sequence of steps, reacting appropriately to situations occurring during the process. For example in task B3, students were required to carry out a serial dilution according to the procedural steps and guided by an illustration, to produce five solutions of specified concentration of IAA, and then proceed to treat samples of seed with the various solutions. Students who were ready to act were able to complete the task (leaving time to complete the task booklet) within the time allowed, safely and economically and without requiring help from the examiner. High scores in the first two and the last assessment objectives in Table 3 (all over 80%) indicate high levels of achievement in the areas of perception and readiness to act. Students were able to focus and take in instructions (choose, locate, select and use items of equipment and chemicals) and proceed safely and efficiently. However some of the shortcomings identified in this domain included students selecting incorrect items of glassware and using chemicals wastefully. Guided Response, Mechanism and Complex Overt Response In each session students were required to carry out complex tasks based on mandatory practical activities that they had previously carried out, and use skills or mechanisms at which they had become proficient by practise. The laboratory techniques demonstrated by the students, e.g. preparing pondweed stem properly before immersion in NaHCO3 solution, preparing microscope slides of biological samples, applying coverslips and stain to the samples, focusing a microscope at high power, dissecting a heart efficiently, and the various procedures of aseptic technique were guided responses or mechanisms that were achieved by imitation, trial and error and repetition before the trial. Students demonstrated competence in guided response and mechanism in the trial by responding appropriately, without detailed instructions, to stimuli in the tasks, e.g. task B4, (Apply the coverslip and stain appropriately to the sample, and, Repeat observation under medium and high magnifications); tasks B5.2, B6, and B7, (Prepare your enzyme source); task B6 (Record the pH using a suitable method); task B10 (Dissect the heart appropriately to clearly reveal its internal structure; and Identify the location of the entrance of the coronary artery); and task B12 (Take five samples of leaf discs from the leaves using aseptic technique). The high score in the second assessment objective in Table 3 and the somewhat lower score in the third

23

indicate reasonably high levels of achievement in the areas of guided response and mechanism. However, various errors in technique were observed by examiners in all schools. These included attempting to focus the microscope starting with the high power objective lens; not applying coverslips properly, leading to large accumulations of air bubbles around the specimen; failing either to blend or finely chop the plant material from which enzyme was to be extracted; obvious unfamiliarity with pH meters and cork borers; clumsy dissection efforts; and poor grasp of aseptic technique. Some candidates were observed to have performed their tasks very quickly and accurately and in a highly coordinated manner. However no distinction was made in direct assessment between a high level of practical achievement and complex overt response, i.e. an expert performance. Adaptation and Origination The trial was based on the mandatory practical activities in the current syllabus. For some students the method referred to in their task booklets may have been different from that used by them in school. For instance some schools employ the method of immobilising leaf fragments or microscopic plants in gel beads for the photosynthesis investigations, B1 and B2. Some students may have been more familiar with amylase rather than catalase for the enzyme investigations in tasks B5, B6 and B7. For the osmosis investigation in task B9 many teachers have taught their pupils to use potato cylinders rather than Visking tubing. In each case adaptable proficiency was required by the students, and all students were required to modify their approach to troubleshoot situations that did not go to plan. As students of the current syllabus are not expected to carry out unseen experiments on their own, the trial did not assess students’ ability to design and execute new approaches that would apply acquired skills to a new practical situation, e.g. using chemicals not previously encountered. 7.2.2 Cognitive Domain

Knowledge and understanding Some short response questions in most tasks required recall of facts or of basic biology terminology, e.g. tasks B1 & B2, iv, (What are the bubbles made of that the Elodea releases?) and task B5.1, ii, (What is the name of the test for proteins?). Many of these short response items were designed to be accessible to students of all levels, e.g. in task B4, iv, students were asked to name the reddish-brown solution, that appears yellow when diluted, that turns some plant cells black. Most students answered these short response items correctly. The tasks also required students to demonstrate their broader or deeper knowledge and understanding of the principles and theories underlying the practical tasks. These interrogations usually took the form of asking students for the reasons behind particular steps in the procedures of various tasks. For example in task B5.1, i, (Suggest why some chemical tests require heating and others do not); and task B9, iv, (What is the purpose of the Visking tubing containing the distilled water in this task); and again in tasks B1 & B2, iii, (Why would using a terrestrial plant not be advised in this investigation?). These items were more discriminating than the short response items referred to above. Most students

24

answered the procedural parts of these questions correctly but not all were able to provide the correct explanations. Understanding of the application of scientific method was widely explored in the Biology tasks in the form of the interrogation of the concepts of variables, constants, replicates, and controls. For example, in tasks B1, B2, B3, B6, and B12, students were asked to identify particular categories of variable or to comment on why only one variable at a time must be allowed to vary. Students’ knowledge and understanding of controls was assessed in tasks B1, vii, c; B3, iii, c; B5.1, iii; B6, iii, c; B12, iii. In this context students were also required to identify sources of uncertainty in the experimental design (tasks B2, ix; B9, ii) and/ or to suggest ways in which their results could be made more reliable (tasks B3, ii; B5.2, vii; B7, ix; B11.1, v; B12, iv) or more accurate (task B1, ix). In task B11.1, students’ understanding of the idea of randomness and its importance was assessed (ii. Random sampling is important in this task. How did you ensure the sample selected was random?; iii. Why must the samples be random?). In B11. 2, i, Students were asked Why was the same number of seeds added to each dish? Generally, such questions were poorly answered, although questions about controls were better answered than those about variables, replicates, and uncertainty. The trial provided an opportunity to include items to assess knowledge and understanding by asking students to record, using suitable terminology, their own observations when carrying out tasks, and then explain them. Examples of this include tasks B1 and B2, in which students were asked, having followed the instructions, to investigate the effect of either different concentrations of CO2 or different light intensities on the rate of photosynthesis, to Record your results in the table below, and, Describe the trend indicated by your results. Similarly, in task B9, having conducted an experiment to investigate the influence of solute concentration on the movement of water through a selectively-permeable membrane, students were asked to complete a table with details of how a Visking tubing preparation had changed in appearance and mass. They were then asked to comment on both the significance of the mass of the tubing containing a hypertonic solution and one containing a dilute solution, and subsequently to Provide an explanation for this observation, in each case. This is a very appropriate way of assessing practical work in Biology which is not available in Section B of the current written examination where the only options are to give an observation and ask candidates to explain it or to ask candidates to recall observations they made earlier in mandatory practical activities and explain them. Unfortunately, in tasks B1 and B2 the experience of most schools was that the pondweed was of such poor quality that students did not manage to generate their own results and had to resort to the pseudodata. In task B9 students generally were able to record their results and most were able to explain them satisfactorily. Application and Analysis In Biology tasks B1, B2, B5, B6, B7, B9, B10, and B11.1, students were required to record data generated by their own work. In tasks B1, B2, B6, and B7, they were to graph these data and use the graphs to establish trends, to obtain optima, and in one instance (task B1, v, If you were to increase the temperature of the waterbath to 30 °C, how would this affect the rate of production of bubbles? and, Show this on the graph you have already drawn), to speculate on extrapolation. Examiners reported that many students were able to draw good graphs of their own data or of the pseudodata, and that the better the graph, the better the subsequent analysis.

25

Students were also required to base calculations on their own data e.g. task B2, x, Calculate the light intensity when the lamp is 60 cm from the pondweed; task B4, vi, A student measured the width of an animal cell to be 24 mm when it was magnified × 400 with a light microscope. Calculate the actual width of the cell; B11.1, i, Using the results obtained, calculate the population of the animal species (counters) (having been given the relevant formula). Examiners reported that the light intensity calculation was either very poorly done or not attempted at all, while the other two calculations were handled well by some students and considerably less so by others. This was to be expected given that the tasks were set at a common level. Synthesis and Evaluation In most Biology tasks students were required to evaluate their own data and decide whether to work on their own data or request, subject to possible penalty at the direct assessment stage, pseudodata. This necessitated their evaluation of the quality of their own work. The penalty was not applied if the examiner deemed the data problem not to be of the student’s making, e.g. where hydrogen peroxide solution had previously decomposed or the pondweed was clearly unresponsive. Students could subsequently be awarded full marks in the indirect assessment for correct evaluation of their own data (even if flawed) and of the pseudodata. The assessment of some practical activities from the current Biology syllabus is restricted in a way that is not the case for either Chemistry or Physics. The mandatory practical activities in ecology, for instance, cannot be assessed in a 90-minute laboratory situation. Neither can some of the activities that require significant growing time for either seeds, more mature plants, or microorganisms. In tasks B3 (the influence of IAA on plant growth), B11.2 (seed germination conditions), and B12 (growth of leaf yeasts), students were assessed on their ability just to set up the necessary conditions for the subsequent growth of the organism in the investigation, and no data could be generated, so no data other than the pseudodata could be evaluated. Nevertheless, students were still asked to evaluate data in such speculative situations e.g. task B12, iii, having been given pseudodata from a hypothetical investigation into air quality involving the growth of leaf yeasts in different locations, students were asked What can you determine from the above results about the air quality in the two locations? Justify your answer. In the Biology tasks the assessment of the higher order skill of synthesis was largely confined to investigating the extent to which students could identify ways in which the particular practical activity in the task could be improved by making it more reliable or its results more accurate. The tasks in which this was assessed were B1, ix; B6, ix; 11.1, v; and B12, iv, and was answered well by some students and poorly by others. In task B8, students had to follow appropriate procedure to isolate DNA from plant tissue. One of the steps involved adding the plant material to a mixture containing washing-up liquid in order to disrupt cell membranes. In part vi of the indirect assessment, students were told It is important not to use an anti-bacterial washing up liquid in this experiment, and then asked Can you suggest any reason why this is so? Examiners reported that very few students answered this correctly.

26

8. Findings

Key messages in relation to the practical assessment and future practical asessment General • Students’, biology teachers’ and school managements’ responses to the notion of

practical assessment of senior sciences and to the model trialled were very positive.

• There is general agreement that teaching and learning of Biology would be enhanced by the introduction of practical assessment and that students would have greater sense of involvement in and ownership of all the practical work they do.

• Students who engage well with practical work and acquire good practical skills presently

receive no credit for their efforts. • There is a perception that at the moment some schools do not carry out some or all of

the mandatory practical work and that their students still perform well in the Leaving Certificate examination. This discourages some teachers from investing a lot of time and effort in practical work and some students doubt the importance of the practical work in supporting them in understanding theory.

• At present practical biology classes are taken less seriously even by ambitious students

than theory classes. Even though students may enjoy practical work, they are not incentivised to work skilfully, to work independently, to get meaningful results, to persevere when things go wrong, to consider extensions to the experiments being conducted or to devise improved methodologies to do them.

• At the moment some schools seem to put more emphasis on and effort into involving

students in performing heart dissections and supervising opportunities to use microscopes to attract potential First Years at the annual open evening than into equipping and organising the school laboratories to service the teaching and learning of the practical work that underpins the theory of the science subjects at junior but particularly at senior cycle. The introduction of practical assessment would bring the emphasis back to scientific enquiry, skills acquisition and purposeful science practice.

• Practical science will help students acquire transferable skills of observation and

measurement, manual dexterity, and a good attitude to safety based on a balanced approach to hazards and their associated risks, that can be applied outside the laboratory.

Designated Support Teachers (DSTs) and Laboratory Preparation • DST support of practical assessment before and during the assessment is critical.

• DSTs in the trial engaged very positively and were very supportive of the role of the

examiner.

27

• Practical assessment in the future would require a set of suitable storage containers and a place to store them securely in advance of the assessment. Much improvisation was observed in the trial.

• The volumes of solutions specified in some tasks could be reduced for environmental

reasons, to economise on use of chemicals, and to save storage space and deionised water consumption.

• The introduction of practical science would require a more planned approach by the

schools’ science departments to laboratory and storeroom management, ordering and stock control of chemicals and glassware, access to laboratories.

• Introduction of practical assessment would require continuous professional development

for teachers with training in safety, chemical storage, etc. Examiners and Students • In order to avoid system failures, the examiners would need a time allocation in each

school before the commencement of the practical sessions to check that all solutions were prepared to specification and were available in sufficient quantity. Schools should be responsible, at short notice, for acquiring missing chemicals from neighbouring schools and preparing further stocks where shortages are identified.

• Task-specific lists of observable skills and opportunities to assess measurement skills should be provided to examiners.

• Tasks should be tested by setters. • Examiner training would need to involve a practice session to familiarise examiners with

the tasks. • A student from a non-exam class acting as a helper (to call DST, etc.) during examining

sessions is recommended.

• Instructions should allow students to write, underline, etc. during the first ten minutes, e.g. to make a list of glassware, chemicals they should bring to their work station.

• While it is possible to assess 12 students in a session, it is difficult to observe repetition of

measurements by any one student in a group of 12. If one or more students required a significant amount of help, the examiner would be under pressure to provide such help and observe the work of the other students thoroughly. The session could be further compromised if there were a spillage or breakage. Examiner guidelines would have to be provided about maximum time that could be spent helping a student in a session.

• Students should be required to label any vessel into which they place a chemical on their workstation to enable the examiner to assess their practical work easily without having

28

to converse with them. Labelling to be done with glass-marking pen, e.g. a Sharpie pen, that could be easily wiped off at the end of the session.

• Students in the trial were reported to have engaged very positively with the carrying out

of their tasks and the majority remained for the full session although they understood that they would receive no credit for their participation and would not get a result or any other feedback on their performance.

• In an actual practical assessment it is likely that students would be more nervous.

However, it is also likely that they would have prepared more thoroughly for the assessment.

• If the direct assessment was accepted as and understood to be high scoring for most

students while the indirect assessment was more discriminating, it would encourage confident performances by students.

• On introduction of practical assessment for all schools, it is recommended that the first

session in a school should not involve more than 8 students. Later sessions could involve up to 12 students.

• Where a student in a session had a catastrophic experience of their own making or as a

result of an error by another student, e.g. a collapse of assembled apparatus, or a breakage that involved a spillage of a solution that could not be quickly dealt with, say within 5 minutes, it should be possible to reschedule the student to a later session (doing another task) to avoid claims for systems failure. The examiner could make a judgement about whether the student should start the later session carrying a penalty of 4 marks for an error or not.

29

chief examiner’s report - biology · trial of practical assessment in leaving certificate...

Documents