guidance and information for teachers 3rd edition...

Guidance and information for teachers

Digital3rd Edition

of 26Copyright © 2013 GL Assessment Limited

NGRT Digital Test Guidance and information for teachers

NGRT Digital

Introduction

The New Group Reading Test (NGRT) was published in a paper edition in September 2010. It comprised four levels of tests with equivalent forms for students from age six to 16.

This digital edition departs in two significant ways from the paper test on which it is based:

1. it is adaptive;2. it comprises two equivalent forms (A and B) which may be administered to students

across the age range seven to 16.

Results from trialling the digital test with six year olds demonstrated that the method of answering the questions was a barrier to such young children and their performance was significantly lower when tested with the digital edition rather than the paper edition. NGRT Digital, therefore is suitable for the following year groups:

NGRT Digital Form

Year GroupEngland and Wales Scotland Northern Ireland

A Years 2–11 P3–S5 P3–Year 12

B Years 2–11 P3–S5 P3–Year 12

Adaptive testing with NGRT

The NGRT digital edition comprises the unaltered content of the paper tests. However, this is presented in a way that allows students’ reading to be tested according to their performance as they are taking the test rather than by age or year group. An analysis of the difficulty of all questions in the paper tests has been carried out in order to create the adaptive, digital test. Form A and Form B of the digital edition mirror versions A and B of the paper tests and content from Test 1 is common to both forms.

Like the paper edition, NGRT Digital comprises three sections: phonics, sentence completion and passage comprehension. However, the digital test is adaptive and each student’s performance is assessed as they complete the test and the questions adapted to be in line with the ability they demonstrate. This is a benefit, particularly to students with weak skills – they can be tested with material at a lower level than that determined by age – and for students with high level skills as they can be tested with material that better reflects their ability.

Students enter the test according to their age. All start with the sentence completion section. Depending on the score from this part of the test, they will be moved on to the passage comprehension section or be given the phonics tasks. Those who complete the phonics tasks will not be presented with the passage comprehension. Testing time will be approximately 30 minutes. However, please note that as NGRT Digital is adaptive the time taken to test each student will vary, so some will finish before others.

All students require headphones to access this test.



The sequence of questions in the sentence completion section is based on the probability of a student answering a question correctly based on those questions answered correctly on the basal set (which is determined by age). This will make testing shorter and, importantly, place the child at an appropriate level for the hardest part of the test, the reading comprehension.

Most students will complete the sentence completion section followed by the comprehension section and read and answer questions on one, two or three passages at the appropriate level of difficulty. If students perform poorly on the sentence completion section they will complete the phonics section and will not be administered any comprehension passages.

Each form of the digital test should be considered as a single test with several parts:

• Sentence completion (a bank comprising all questions across all levels)• Phonics section (the full content from Test 1)• Reading comprehension – passage by passage rather than grouped by level as

students will be directed to an appropriate initial passage on the basis of performance on sentence completion. Generally, this should be a secure basis for selecting the next part of the test but some students’ skills may be stronger or weaker in sentence completion than in passage comprehension so the program will adjust to performance on the initial passage too.

So:

• All students will start the test with sentence completion• The program will determine the first question based on date of birth and each student

will progress through the test according to performance• Students with very low scores will move to the phonics section• Students will be presented with one, two or three passages; this will shorten test time

and is possible because the level of passage will be suited to the individual on the basis of their sentence completion score.

Administering the test

Note: The test administrator will need to read through the following instructions well in advance of the test session.

Prior to testing

It is strongly advised that you go to:

www.testingforschools.com

and click on Setup Check where you will be able to run the System Compatibility check. This will confirm that your system is compatible with GL Assessment digital tests and will ensure the smooth running of the test session. (The complete Help screen can be accessed once you have logged in with your Customer ID.)

You should have been sent your Account URL in your ‘Welcome’ email:

www.testingforschools.com/#/Customer ID/



By clicking on this link, students can access the option to ‘Take a test or Complete a survey’. All students participating in the test must be added to the class register for NGRT Digital on Testwise in advance.

On the day of the test, students should click on the icon ‘Take a test or Complete a survey’, type in the Register ID and select their name from the register. At this point they should type in their password provided by you, their teacher or an administrator. If the children are very young, the teacher or administrator may prefer to log on for each student.

Alternatively you can provide a URL with the Register ID in the following format:

www.testingforschools.com/#/Customer ID/Register ID/

This will take the students directly to the list of names on the register.

The students will see the test as allocated to them and should click on the Take Now icon. They will then be able to start taking NGRT Digital.

If, on the day of testing, you experience any difficulty please contact the Technical Support team on 0845 602 1937 (Option 2) or email [email protected].

Administration

The test must be administered in a formal test environment with students made aware that they are taking a test and that the usual expectations of behaviour and constraints of a test session will be in place.

Each student will need their own personal computer with a mouse and a high quality set of headphones. You are advised to check that headphones are working and that sound levels are appropriate before logging on to Testwise.



All administration instructions are incorporated into the test program but please introduce the test session and check that students understand the instructions and the method of answering by walking around the room as they work through these parts of the test.

When students are settled, explain that they are going to take a test that will show how well they can read and understand what they have read. Students must work in silence but if they have a query they should raise their hand and wait for the teacher to approach them. Answer any questions at this stage and explain that you cannot help with any of the test questions.

Reassurance

Further explain that the test starts with some practice questions to get them used to the real questions and that all instructions are given by audio. NGRT is not a timed test so students should work through the questions at their own pace.

Sentence completion – example questionAll students will start the test with the sentence completion section and see the following example question.

The voice over will explain that the students have to click on the word (from a choice of five) that they think best completes the sentence.

Once the students have clicked on ‘Next’ they cannot return to the previous question.

In the practice section students will receive feedback telling them whether they have answered correctly or incorrectly. This will not be a feature of the test, however.

Because the digital version of NGRT is adaptive, each student will see a different set of test questions. Performance on the sentence completion section will determine which part of the test will be administered next.



Phonics section – example questionsThis section of the test is in four parts and each part has a single example question with feedback.

Initial letter itemsThese questions are supported by a picture and the student is asked to identify the first letter of the word illustrated (the audio provides the word).

‘Sounds like’ itemsAgain, supported by a picture and by audio, the student is asked to select the word that sounds like the item illustrated.



Final letter sound itemsStudents are presented with the first part of a word and from a selection of digraphs or trigraphs (two or three letters which together represent a single sound) and three letter combinations have to complete the word.

Initial letter sound itemsStudents are presented with the final part of a word and a selection of letters from which they must select one to complete the word.

These four short exercises are aimed at students who are at the very beginning of learning to read or whose reading is delayed and there is a need to test that they can match letters to sounds, understand simple rhyme and complete real words by adding final or initial sounds.



Passage comprehension – example questionsStudents will see one, two or three passages (determined by performance on sentence completion and, as appropriate, the initial passage). A short passage, appropriate to the demands of the test passage is presented as an example and for practice.

For the easiest passage comprehension, ‘The Park’, a short example passage about looking after a pet mouse replicates the layout of the real test questions where the story text and questions relating to that part of the story are on the same screen. Here is an example:

Here is an example of one screen from ‘The Park’:



All other passage comprehension tasks are introduced by a short passage with example questions which show students how to answer the questions. In these passages students can navigate through the story text using the scroll bar and through the questions using the ‘Next’ and ‘Back’ buttons separately so they can refer to any part of the story at any time to answer any question.

The example section looks like this:

Here is an example from one of the test passages:



During the test

During the test session, you should walk around periodically to prevent copying and to ensure that students understand how to answer the test on-screen. Any students showing signs of difficulties should be given assistance. However, you should give no help at all in the choosing of answers or explaining text.

Testing time will be approximately 30 minutes. Some students may finish in a shorter time and should have work to hand so that they remain occupied until the whole group has completed the test.

At the end of the test all responses will be stored on the Testwise server. Please do not turn off any computer until the students have clicked on the End Test button and have been taken back to the page showing that the test has been taken:

The test environment

As mentioned above students completing NGRT need to be in a quiet room without interruptions. Enough time should be allocated to allow students to settle, be given the introduction to the test and complete it. A total of 40 minutes should be sufficient to introduce and administer the test.

Students should be told that they must work in silence but that if they have a query they should raise their hand and wait for the teacher to approach them.

While students are completing the test, the teacher should walk round the computer suite to check that they are progressing appropriately and that they are not having difficulty with the methods of answering questions.



Unexpected incidents

If there is a failure in your computer system while students are completing the test, it will not be possible to re-enter the test at the point at which the failure occurred. If students have fully completed a section of the test and a computer failure occurs, they will be able to recommence the test at the section after the one they have completed. If students complete the test and responses are stored (that is, they have clicked the End Test button) and then the system fails, it will be possible to retrieve responses and reports from the GL Assessment back-up server.

NGRT Group report

A group report will be available after testing and is made up of the following sections:

• OverviewThis is a short introduction which describes the test content and how the test works. This will be useful when sharing reports with colleagues who may not be familiar with NGRT.

• Why use NGRT?Again, this is a short section setting out the main uses of NGRT.

• Relationship between NGRT scoresThis comprises a simple diagram showing how standard age scores (SAS), stanine scores and national percentile ranks relate to each other.

• Example resultsThis page shows an annotated version of the group report table with a glossary of terms used.



• Group scores for sentence completion and passage comprehensionScores for the pre-determined group include SAS, stanines, group rank and national percentile rank. Importantly, this table highlights a gap of two or more stanines between the Sentence Completion and Passage Comprehension parts of the test.

• Analysis by reading ageThis shows the spread of reading age for the whole group and for males and females.



• Group scores for phonicsThis shows raw scores by section for students who answered the phonics questions.

• Group demographic comparison with gender as the baselineThis compares the group’s score for each gender against all students as part of the group average and the national average.



NGRT Individual report for teachers

This report focuses on results for an individual student. It opens with the sections:

• Overview• Why use NGRT Digital?• Relationship between NGRT Digital scoreswhich are very similar to those in the NGRT Digital Group report.

Example results are annotated.



The NGRT Digital Individual report for teachers offers an analysis of responses to both the sentence completion and passage comprehension parts of the test and an analysis of the types of comprehension questions achieved by each student.



The page above shows the full set of scores and a graphical representation of the results from the sentence completion part of the test.

All questions have been put onto a scale representing the difficulty of each question and the same scale has been used to establish the reading ability level of an average student at any given age. Thereby, a student begins the sentence completion test with a question that is slightly below the difficulty level that would match the age-related reading ability level. This is shown on the graphic about as ‘Question difficulty’ and ‘Reading Ability scale’.

The test adjusts quickly to present questions of either increasing or decreasing difficulty depending on whether the student answers correctly or incorrectly.

At the end of this part of the test, the student’s reading ability is re-calculated and this new Reading Ability Scale score determines which part of the NGRT Digital will be administered next.



The page above shows the graphical representation of responses to the passage comprehension part of the test. The same ‘Question difficulty’ and ‘Reading Ability scale’ have been used. In this part of the test, students must read and complete each passage before moving onto the next one; that is, the passages are not adaptive. However, after the initial passage, the student’s reading ability is re-calculated and an appropriate next passage is administered.

Comprehension questions have been classified and an analysis of which questions have been achieved by the student is shown in the table.

Finally, this report includes a short narrative which will highlight the need, if any, for further assessment and recommend an appropriate next step.



Where students have been administered the Phonics part of NGRT Digital, these scores are given in a table with a short narrative which takes into account both the age of the student and the test results.

NGRT Digital Group progress report for teachers

The third in the suite of NGRT Digital reports is the Group progress report for teachers. This allows the results from two administrations of the test to be compared for a group of students. A six month time period between test sessions has been applied as the default; this will allow for both progress to be made and for the test to measure this progress in a meaningful way. Re-testing over a short time frame can be counter-productive and the results may not be reliable.

Please note that if you are using NGRT twice in the same school year you must administer both forms (in either order) to be able to generate this report. Form A and Form A or Form B and Form B should be used over two or more school years but not within the same school year.

The report has the following sections:

• What is NGRT Digital?• Why use NGRT Digital to track progress?

both of which give background information and an explanation of why the report has been developed.

The third section:

• What is the Reading Ability Scale?

explains how progress can be quantified using an additional measure which is unique in this format to NGRT.



Example scores are annotated:


Example ResultsThe Standard Age Score (SAS) is the most importantpiece of information derived from NGRT. The SAS isbased on the number of questions a student hasanswered correctly: the score is adjusted for age andplaced on a scale that makes a comparison with anationally representative sample of students of thesame age across the UK. The average score is 100.The SAS is key to benchmarking and trackingprogress and is the fairest way to compare theperformance of different students within a year groupor across year groups.

The NC (NationalCurriculum) readinglevel is based onteacher assessmentcollected when the testwas developed. It is anestimate of the level thestudent has attained atthe time NGRT wasadministered.

Performance on a test like NGRT can beinfluenced by a number of factors and theconfidence bands are an indication of therange within which a student's score lies.The narrower the band the more reliablethe score. 90% confidence bands are avery high level estimate.

The Stanine places thestudent's score on ascale of 1 (low) to 9(high) and offers abroad overview of his orher performance.

This is the differencebetween theStandard Age Scorefor the firstadministration of thetest and the secondadministration of thetest.

Reading ageconfidence bandsStudent name Age at test

(yrs:mths)OverallStanine

Upper

StanineNC readinglevel

PCReading age

SC

SAS (90% confidence bands)NPR

LowerSAS SAS

difference Progress category ReadingAbility Scale

Marianne Gilbey6:09 1217:06 120

10:8928 9:4 3B10:0 6 7304

11:8918 10:2 3A10:11 9 8320-1 Average

John Smith12:01 9812:11 119

12:4455 10:10 4C11:7 5 5332

17:0+908 16:0 5A17:0 8 8408+21 Above average

The National Percentile Rank(NPR) relates to the SAS andindicates the percentage ofstudents obtaining any score. NPRof 50 is average. NPR of 5 meansthat the student's score is within thelowest 5% of the national sample;NPR of 95 means that the student'sscore is within the highest 5% ofthe national sample.

The readingage (or ageequivalentscore) is theage at which ascore is mostlikely to beachieved basedon the nationalsample.

Age at test is thechronological ageof the student ateach test point.

To allow for an easycomparison between astudent's ability inSentence Completionand PassageComprehension, aStanine score is givenfor both parts of NGRT.A gap of two or moreStanines is highlightedin the report.

Progress has beendefined as belowaverage, average orabove average and isbased on the nationaldata set of studentstested on twooccasions. Nationally,15% of students willfall into thebelow/above averagecategories and 70% inthe average category.

A student's score isonly an estimate oftheir ability on any oneoccasion asperformance can beaffected by a numberof factors. The dotrepresents thestudent's SAS and thehorizontal linerepresents theconfidence band. Theyellow shaded areashows the averagescore range.

The Reading Ability Scaleis a development scale andcan be used to monitor astudent’s reading ability ordevelopment over time. Thevalues on the scale are 0 to600 – a higher scale scorerepresents a higher readingability level. The readingability of a student aged 6years is around 200; for age9 years it is around 300; andfor age 16 it is around 400.

Copyright © 2013 GL Assessment Limited

The table of Group scores allows immediate comparison of each student’s performance at both test points. Direct comparison can be made of:

• Standard Age Score• Overall stanine• National Percentile Rank• Age Equivalent Score• NC Level• Stanine for sentence completion• Stanine for passage comprehension

In addition, a visual representation of progress is included by the graphic showing the Standard Age Score with confi dence bands.

Level of progress is shown by SAS points gained (or otherwise) and this is further described as below average, average or above average.

If a student’s reading is average the SAS will be between 89 and 111. Because NGRT results are adjusted for age, average progress over a period of six months or more will be demonstrated by a similar score at both test points. A slightly lower score at the second test point does not mean that progress has not been made; the gap has to be statistically signifi cant to show lack of progress or, indeed, higher than average progress.



Some examples:

Connor Callaghan is a good reader and has demonstrated average progress over the 12 months between test points. This can be seen very clearly from the graphic which shows that his performance on both occasions is more or less identical: A drop of 2 standard score points is not significant.

The drop in stanine score for sentence completion is probably due to test content at a higher level. This is determined by the chronological age of the student so will represent 12 months’ higher difficulty level from the first question with the test adapting to a level that reflects Connor’s reading ability after two or three questions.

Ruth Galley is an above average reader and again, like Connor, her results are, more or less, identical. However the gap between Ruth’s reading at sentence level and at passage level has widened. This may be something that needs attention although with stanine scores of 5 and 9 (sentence completion and passage comprehension) her performance is in the average and high range respectively.

Philip Lima has made progress that is classed as below average with a score at the second test point that is 8 standard score points lower. His reading overall is now in the below average range. Of concern will be the stanine score of 2 for passage comprehension. Philip’s sentence level reading has improved and as performance on this first part of NGRT determines the difficulty level of the first passage, this may have impacted on his performance in passage comprehension.

Thomas Hogarth’s reading has improved to the extent that he is now in the average range and his progress can be classed as above average. His age equivalent score is much closer to this chronological age too. In both aspects of reading tested by NGRT Thomas has improved.



Analysis by reading age equivalent band

No. of students11:00–11:11 12:00–12:119:00–9:118:00–8:11

Mean age(yrs:mths) 14:00+6:11 or less 7:00–7:11 13:00–13:11

Percentage of students by reading age equivalent bands

10:00–10:11


NGRT Form(s): A

School: Sample School

No. of students: 50

Date(s) of first test: 29/06/2012 – 14/09/2012

Group: ST

Date(s) of second test: 04/06/2013 – 18/06/2013 NGRT Form(s): B

All students 5011:04 4% 8% 8% 12% 0% 12% 20% 6% 30%

12:03 0% 0% 10% 10% 10% 2% 4% 12% 52%

Males 2511:04 2% 6% 2% 6% 0% 6% 10% 6% 12%

12:03 0% 0% 6% 6% 0% 2% 4% 8% 24%

Females 2511:04 2% 2% 6% 6% 0% 6% 10% 0% 18%

12:03 0% 0% 4% 4% 10% 0% 0% 4% 28%


The report includes an analysis by reading age for the group. The distribution of age equivalent scores has changed for the whole group and for boys and girls. Of note in the example is the increase in the percentage of boys in the top band (+12%) after 12 months.




NGRT Form(s): A


No. of students: 50


Group: ST


Group demographic comparisonAll students analysisThe table below shows mean (average) scores for your group and the distribution of scores for your groupcompared with those for the national sample at the first point of testing followed by the second point of testing.

MeanSAS 89–9674–81 97–103 >126119–12682–88No. of

students 112–118104–111<74

832 51 64 97Stanine

17% 4%20%17%7%100 7%12%4% 12%–National

All students 50101.1 2% 14% 8% 8% 16% 20% 26% 2% 4%

106.1 0% 10% 10% 12% 6% 18% 20% 18% 6%

SAS (90% confidence bands)All students No. of students Mean SAS

All students 50101.1106.1

National – 100.0

The table below shows the mean (average) scores with confidence bands for your group.


Set out is the distribution of SAS and stanine scores for the group against the national average.



There follows a similar analysis by gender. Further analysis will be determined by the demographic information uploaded for the group. Here is an example of this group analysed by Free School Meals.


NGRT Form(s): A


No. of students: 50


Group: ST


Group demographic comparisonFree school meals analysisThe table below shows mean (average) scores for your group and the distribution of scores for your groupcompared with those for the national sample at the first point of testing followed by the second point of testing.

MeanSAS 89–9674–81 97–103 >126119–12682–88No. of

students 112–118104–111<74

832 51 64 97Stanine

17% 4%20%17%7%100 7%12%4% 12%–National

All students 50101.1 2% 14% 8% 8% 16% 20% 26% 2% 4%

106.1 0% 10% 10% 12% 6% 18% 20% 18% 6%

No 40102.6 2% 10% 4% 6% 16% 14% 22% 2% 4%

108.1 0% 6% 8% 8% 4% 14% 16% 18% 6%

Yes 1095.3 0% 4% 4% 2% 0% 6% 4% 0% 0%

98.0 0% 4% 2% 4% 2% 4% 4% 0% 0%

SAS (90% confidence bands)Free school meals No. of students Mean SAS

All students 50101.1106.1

No 40102.6108.1

Yes 1095.398.0

National – 100.0

The table below shows the mean (average) scores with confidence bands for your group.




Progress profiles


NGRT Form(s): A


No. of students: 50


Group: ST


Males

The NGRT SAS scores for the first andsecond administrations of the test are shownin the diagram. Students who areconsidered to be making average progressare in the white band. Students makingbelow average progress are in the orangeband and those making above averageprogress are in the green band.

Above average progress

Average progress

Below average progress

Females

•


The final section of the report presents data in a scattergram showing the distribution of categories of progress for the group. From this example it is clear to see that most students are above the dashed, diagonal line and this represents that the majority has made progress across the ability range. This is confirmed by the table which, in this case, shows that all but 4% of this group have made average or above average progress.



The last pages of the report list students in each category of progress.



Comparing students’ performance on sentence completion and passage comprehension

NGRT provides a standard age score derived from the test as a whole which gives the most accurate and reliable indication of a student’s reading ability. However, it is possible to look at sub-scale scores for each section of the test in order to determine if students have particular areas of strength or areas for further development in their reading skills.

The scores for the sub-scales are presented using stanines rather than standard age scores. Stanines minimise the over interpretation of small, insignificant differences among test scores. As a rough guide, if the sub-scale stanines are the same or differ by one, this indicates similar performance on the two sub-scales. If the stanine score differs by two or more, it may be an indication of significantly different performance on the two sections of the test.

Therefore it is important to recognise that any significant differences in the two subscale scores may only be indicative of a difference in performance and would benefit from further investigation. For example the Suffolk Reading Scale (SRS) could be used to confirm students reading ability at sentence level and the York Assessment of Reading Comprehension (YARC) could be used to confirm their ability in reading comprehension.

Sentence completionThe sentence completion section of the NGRT assesses students’ reading skills at sentence level. These skills are slightly different to those used when reading a whole text and answering retrieval and inferential questions about that text. In the sentence completion questions students have much less contextual information to use in order to determine the missing word. Therefore students who rely heavily on contextual cues to draw even simple inferences may score less well on this section of the NGRT. The lack of contextual cues will mean that students have to rely more heavily on their word knowledge (vocabulary) in order to successfully answer the more difficult sentence completion questions where the vocabulary may be considered challenging for students in that age range. Therefore students with a more extensive vocabulary may perform better on the sentence completion questions.

In sentence completion questions the students also need to rely more heavily on syntactic cues (the structure of a sentence) and their own knowledge of grammatical features when deciding which of the five possible options best completes the sentence. Some of the distracters that are used in the sentence completion questions have been chosen as they are semantically (in meaning) or graphically (in written form) similar to the target word; however using any of these distracters would make the sentence grammatically incorrect.Therefore the student is required to read both the sentence and all five of the answer options very closely in order to determine the correct answer.

Although students need to be able to make single local inferences in order to answer the sentence completion questions they do not need to employ any of the more complex global inferencing skills required in the passage completion section of the test. Therefore a student who has difficulty understanding longer texts or making complex inferences may perform significantly better on the sentence completion section of the NGRT.

Passage comprehensionIn the passage comprehension section students need to use a variety of reading skills to read up to three passages and answer a number of questions on each passage. In reading and answering the texts students are expected to make global as well as local inferences. Each text is accompanied by a range of question types including:



• Context comprehension• Retrieval• Inference and deduction• Organisation of texts• Writer’s use of language• Writer’s purpose and viewpoints• Social, cultural and historic tradition.

The scores students obtain on this section of the test are influenced by their ability across a wide range of reading skills. Students who are very good at decoding but poor at inference and deduction may obtain a significantly lower stanine score on this section of the test.

Early feedback from teachers is that the majority of students will achieve scores that are within one or two stanines: for example 2:3 or 7:6 and for exceptionally good readers 9:8 or 7:9. It is likely that just a small number of students will need additional investigation as indicated above but this aspect of NGRT now enables screening and progress tracking for large groups of students and more meaningful individual profiling from a single test.

In case of further enquiries please contact GL Assessment Customer Services on 0845 602 1937.

Copyright © 2013 GL Assessment Limited.

GL Assessment is part of the GL Education Group.

9th Floor East, 389 Chiswick High Road, London W4 4AL

www.gl-assessment.co.uk

[email protected]

guidance and information for teachers 3rd edition...

Documents