selection assessment services test brochure.pdf · selection assessment services ... • all...

SELECTION ASSESSMENT SERVICES

RESEARCH-BASED QUALITY ASSESSMENTS

FOR SELECTIVE SCHOOLS

At CEM we have been providing assessments for schools for over 30 years, and developing selection assessments since 1999. We provide paper and computer-based selection assessments to schools, trusts and authorities, for children predominantly of ages 10 to 11 years.

3

Our Approach ............................................................................................................................................................ 4

Developing Fair Assessments.................................................................................................................................... 4

What we assess ........................................................................................................................................................ 6

Development, reliability and validity ....................................................................................................................... 7

Assessment Options .................................................................................................................................................. 8

Computer-based assessments – CEM Select ........................................................................................................ 8

Paper assessments ............................................................................................................................................... 9

Bespoke assessments ........................................................................................................................................... 9

What our customers say about our service ............................................................................................................... 10

The Team ............................................................................................................................................................... 11

Find out more............................................................................................................................................................. 11

CONTENTS

4

We strive to make the selection process fair for all candidates, and our assessments are designed to enable all children to demonstrate their natural ability and achievement, without the need for excessive preparation. Our aim is to develop assessments that are taken in as standardised an environment as possible, so that each candidate has the same assessment experience.

In order to maintain the fairness of our assessments we do not provide any commercially available practice materials, nor do we recommend any other commercially available resources.

Developing Fair Assessments At CEM, we are passionate about ensuring that all candidates have an equal opportunity of success. To achieve this, 11+ entrance assessments from CEM are developed under the following principles:

Skill Based ContentOur assessments focus on evaluating the natural reasoning abilities of candidates. Knowledge based assessments are susceptible to tuition and prepping, potentially advantaging candidates with the resources to supplement their education. Research indicates however that underlying reasoning abilities can be tested without the need for introducing excessive knowledge based material. Based on this, CEM selection assessments focus on verbal, mathematical and non-verbal ability only, and do not draw upon national curriculum content beyond that which is compulsory for children aged 10 and below.

Removing BiasWe are a world leader in educational research and understand that not all children learn in the same way. Many groups perform differently in specific skill sets. Our research shows that male candidates tend to show an advantage in mathematical reasoning while female candidates tend to perform better on verbal questions. Candidates with English as an additional language may be disadvantaged through less exposure to reading in English. Children with Dyslexia may have a specific deficit in reading ability but have intact understanding. Our selection assessments are developed with the aim of minimising the impact of these factors, using a range of strategies to achieve this.

OUR APPROACH

5

Resistance to TuitionWe are aware that the practice of intensive tuition surrounding selection assessments is a major issue in the education sector. Concerns are frequently raised that many able candidates may be at a disadvantage, compared with candidates with the resources to prepare for selection assessments. In order to mitigate any disadvantage, our assessments implement the following controls:

• Content is continually refreshed and does not rely on formulaic material.

• New assessment paradigms are regularly introduced to reduce the predictability of assessment content.

• Content is developed independently from all other CEM assessments, ensuring that candidates who have used our existing systems are not advantaged from familiarity with existing assessments.

• We do not provide any commercially available materials (including practice or past papers).

• We do not endorse, support, or work with any third party provider of materials or tuition services.

• All candidates receive standardised familiarisation materials free of charge from their school or local authority. This provides enough familiarisation to prevent undue anxiety for candidates sitting the assessments.

• Assessment delivery is standardised across centres ensuring that all candidates get the same assessment experience.

• No information regarding our assessment development is released to the public.

Whilst CEM acknowledges that no assessment can ever be truly tutor-proof, we are confident that our rigorous controls minimise the potential for tutors to prepare candidates with assessment specific knowledge.

“We strive to make the selection process fair for all candidates…”

6

WHAT WE ASSESS

Our assessments cover three areas of cognitive ability: numerical, verbal and non-verbal ability. Each area is designed to measure a specific skill set. By assessing a variety of skills, candidates can demonstrate their potential even if they are weaker in one area.

Verbal abilityCEM has many years of experience in developing successful measures of literacy and verbal abilities. Our assessments take a broad definition of verbal ability and include comprehension, reading and vocabulary skills, with assessments balanced to minimise bias to any one specific skill sub-set.

Computer Assessment

Page 3

© The University of Durham, as represented by Centre for Evaluation and Monitoring (CEM) 2013

Questions which test Verbal Ability

Find the Missing Words In each of the following sentences, there is a word missing. Please complete each sentence by selecting one word from the options A-E. Shade your answer on the answer sheet.

A B C D Equiet visit people hotel tourist

Buckingham Palace is one of the most famous Example i attractions in London.

The correct answer is E, tourist. The answer E has been shaded in for you on the answer sheet.

A B C D Edemolish notice celebrate nd show

Alisha was organising a surprise party to Question 1 Grandpa’s birthday.

Shade the answer on your answer sheet.

ComprehensionCarefully read through the passage of writing and answer the questions that follow. Shade your answer on the answer sheet by choosing one of the options A-D.

Mother’s Day probably has its origins in Greek or Roman times. In more recent centuries, it has marked occasions when servants were granted an afternoon off work to visit their mothers. A commercial element has added a contemporary twist to the tradition with the advent of Mother’s Day cards.

Example iWhen does the passage suggest that the tradition of Mother’s Day began?

A No-one can guess when it may have started.

B It is a modern invention.

C It is likely to date back to Ancient Rome or Greece.

D Last century, when servants were given time off work to see their mothers.

The correct answer is C. The answer C has been shaded in for you on the answer sheet.

1 What does the author say about the effect of Mother’s Day cards on this tradition?

A They provide a money-making opportunity.

B They have brightened up the celebrations.

C They have twisted the real meaning of Mother’s Day.

D They are not necessary for this historic celebration.

Shade your answer on the answer sheet.

Paper-based Assessment

7

Numerical abilityOur assessments test a range of numerical skills including arithmetic, operations, data handling and functional skills. Sections are balanced to cover as many different elements of mathematics as possible. The emphasis of our numerical ability sections is on testing how candidates solve problems and reason with numerical concepts rather than simple assessments of formal operations and calculation.

Non-verbal abilityResearch has consistently demonstrated that abstract, non-verbal skills are strong correlates of ability and intelligence beyond traditional verbal and numerical reasoning measures. Such assessments are knowledge and curriculum free, difficult to prepare for, do not involve verbal and numerical stimuli and are independent of other cognitive abilities. Implemented non-verbal ability measures are also selected carefully to eliminate gender bias, and thus minimise content focusing on purely spatial stimuli (such as mental rotation).

Development, reliability and validitySelective entrance testing impacts upon thousands of candidates each year. It is therefore important that parents, schools and local authorities have confidence in the assessments we provide. CEM has been successfully developing entrance assessments since 1999, and our assessment materials are developed by a team of educational professionals experienced in the creation of psychometric assessments.

Assessment development is also tailored to the individual requirements of the local authorities and schools using them, and a new assessment product can take one to two years to develop fully before it is released. Requirements are based on admissions policies, catchment areas and the ability of intake cohorts; meaning no two assessments are alike. This ensures that candidates enrolling for multiple assessments across different locations will never be advantaged from familiar content.

Computer Assessment

Page 5


1 In a survey of Year 6 pupils, 25% said their favourite colour was red. If 16 pupils said red, how many pupils are there in Year 6?

Complete the answer on your answer sheet.

2 I think of a number N. I add 5 to N and then multiply the answer by 3. The result is 84. What is the number N that I rst thought of?

Complete the answer on your answer sheet.

A group of Year 6 children conducted a study of vehicles travelling along the road outside their school.

Here are some facts from the study:

• 153 vehicles were recorded in total

• Twice as many vehicles travelled north as travelled south

• No vehicles stopped on the road

• 3 buses were recorded

• 12 lorries were recorded

Use this information to answer the following questions.

For each question, choose one of the options A – E.

3 If the only vehicles that were recorded were cars, buses and lorries, how many cars travelledalong the road?

A B C D E134 136 138 140 141


4 How many vehicles travelled north?

A B C D E73 77 81 84 102


5 This study lasted for 3 hours and the traf c was quite evenly spread over this time. What is the most likely number of vehicles recorded in the rst hour of the study?

A B C D E20 47 71 82 104



Computer Assessment


Page 6

1

2

Questions which test Non-verbal Skills

A B C

D E F

A B C

D E F

A B C

D E F

A B C

D E F


Each test item is trialled and validated on samples matched for age and academic ability. Paradigms and items are examined thoroughly in order to minimise the impact of known sources of bias, before appearing in published CEM assessments. All items and sections are rigorously analysed to ensure only the most reliable measures of ability are implemented, ensuring that our selection assessments consistently discriminate on the basis of natural ability.

Construct and concurrent validity of CEM 11+ assessments are high with standardised test scores demonstrating strong correlations with KS2 performance. Total standardisation scores, for example, correlated with KS2 average point scores as highly as 0.82 in one study.

Users of our assessments can therefore be assured that CEM selection assessments are robust, independent measures of ability that give all candidates the best possible chance of success.

ASSESSMENT OPTIONS

8

By working in partnership with schools and Education Authorities we have been able to develop assessments that consistently perform well year-after-year in accurately identifying the most able pupils. We follow a rigorous design, development, trialling and testing process to produce assessments that are fit for purpose.

Our paper assessments can be tailored for each school or authority. In general we take 1-2 years to fully develop the content and service for an entirely new system. We also provide computer based packages.

Computer-based assessments – CEM SelectOur range of computer based assessments are becoming increasingly popular with a number of Independent and Grammar Schools. CEM Select is available in the following formats:

• CEM Select Challenge provides refreshed content each year, and no national norms are available. This assessment is suitable for entry into Years 7, 8 and 9.

• CEM Select Evaluate provides year-on-year comparability, and national norms are included. This assessment is suitable for entry into Years 7, 8 and 9.

• Mid-Year Transfer provides year-on-year comparability with standardised age norms specific to your school. This assessment is suitable for entry into Years 7, 8 and 9.

What is involved?Having registered and signed a service level agreement, your school can download the assessment software onto your school’s local area network. The assessment comprises several timed sections that cover literacy, numeracy, verbal and non-verbal ability. The assessment is usually completed in under an hour.

FamiliarisationWe recommend that schools share our familiarisation website with candidates and parents. This gives candidates basic information about what the assessment will look like, the format of the questions and how they should be answered:

www.intuproject.org/CEMSelect/CSFamiliarisation.swf

Pupil SupportEach package comes with a 25% extra time version and is available for candidates with special educational needs.

ReportingRaw scores are reported, including sub-section scores and a total score to enable you to rank your candidates. Standardised scores are also provided for some services. The facility to check that candidates have not also taken the test before is also available. Feedback is automated and downloaded from our CEM Plus website.

9

Paper assessmentsWhat is involved?Paper assessments are comprised of either one or two papers which candidates will sit in authorised assessment centres. We divide each assessment into specifically timed sections, to best reflect the balance of cognitive abilities being tested. The delivery of the assessment is standardised by an audio soundtrack. Each new section is introduced using examples as well as informing the candidate of the time they have available.

Extra time of 25% is available for candidates with special educational needs. Modified papers for candidates with visual impairments are also available on request. Other reasonable adjustments can also be implemented in the assessment environment where necessary for other particular needs (such as hearing impairments).

FamiliarisationWe recommend that schools provide candidates and parents with a copy of the familiarisation sheet for the assessment. This gives candidates basic information about what the assessment will look like, the format of the questions and how they should be answered. Familiarisation material is available to centres on request.

ReportingWe provide age standardisation of the results and a breakdown of scores at a task or section level providing the ability to rank your candidates. A total score and sub-scores are provided. No national norms are available.

Bespoke AssessmentsIf you feel that your school could benefit from its own unique assessment, CEM can also support you. CEM has a long and successful history of developing a range of customised assessments specific to the needs of individual schools. These can take the form of computer or paper based tests and can be modifications of existing products, or entirely new, fully customised tests unique to your school. Feedback can also be customised from using simple raw score ranks through to a range of standardised scoring options.

To ensure that new designs and assessments are fit for purpose, bespoke assessment products usually take about a year to

create (although this may vary depending on the scope of your requirements). Please get in touch with our assessment development team if you are interested in discussing further assessment options.

FeedbackFeedback contains all relevant details required to find and examine candidates who have taken a test. Date of birth and test dates are also recorded to allow age standardisations where necessary. In addition, our computer-based products also use this data to check if candidates have sat the same or similar assessments previously in other centres to ensure any potential candidate advantage is highlighted to further inform selection processes.

ID Surname Forenames DOB Age Gender Current School Venue TestDate Year Group Section 1

Section 2

Section 3

Section 4

Section 5

Verbal

Maths

Non Verbal

Total

std Verbal

std Maths

std No

n Verbal

Total W

eightedSc

1001 Apple Andrew 06/08/2004 10.11 M St Anne's CEM 17/09/2014 7 19 28 11 13 32 43 32 32 107 115 132.5 112.4 119.971002 Banana Bob 03/03/2004 10.54 F St Bernard's CEM 16/09/2014 7 23 26 13 16 28 52 28 28 108 127.3 122.6 109.5 119.81003 Carrot Charlie 12/07/2004 10.18 M St Cuthbert's CEM 16/09/2014 7 14 21 12 11 32 37 32 32 101 99.1 101.4 112.4 104.31004 Damson David 02/10/2003 10.96 M St David's CEM 16/09/2014 7 17 18 7 9 22 33 22 22 77 84.4 95.4 98.6 92.81005 Fig Ferris 14/12/2003 10.76 M St Francis' CEM 16/09/2014 7 11 21 9 7 22 27 22 22 71 80.5 101.4 98.6 93.5

ID Surname Forenames DOB Age Gender Current School Venue TestDate Year Group Section 1

Section 2

Section 3

Section 4

Section 5

Verbal

Maths

Non Verbal

Total

std Verbal

std Maths

std No

n Verbal

Total W

eightedSc

1001 Apple Andrew 06/08/2004 10.11 M St Anne's CEM 17/09/2014 7 19 28 11 13 32 43 32 32 107 115 132.5 112.4 119.971002 Banana Bob 03/03/2004 10.54 F St Bernard's CEM 16/09/2014 7 23 26 13 16 28 52 28 28 108 127.3 122.6 109.5 119.81003 Carrot Charlie 12/07/2004 10.18 M St Cuthbert's CEM 16/09/2014 7 14 21 12 11 32 37 32 32 101 99.1 101.4 112.4 104.31004 Damson David 02/10/2003 10.96 M St David's CEM 16/09/2014 7 17 18 7 9 22 33 22 22 77 84.4 95.4 98.6 92.81005 Fig Ferris 14/12/2003 10.76 M St Francis' CEM 16/09/2014 7 11 21 9 7 22 27 22 22 71 80.5 101.4 98.6 93.5

Example Feedback Data

Test data is returned in the form of raw scores for individual sub-sections, full sections and test totals. Where a standardisation has been applied, standardised section and total scores immediately follow. Standardisations can be cohort specific or normatively referenced and can also be standardised by age depending on the product. The total scores are usually weighted from each section.

Where a standardisation has been applied to the data, colour indicators are added to aid visual inspection and allow you to examine the ability profile of each candidate.

Feedback can be provided in a variety of different ways (including changes to standardisations and test weightings) and should you require something different, please get in touch with our team to discuss potential options.

“…we have been able to develop assessments that consistently perform well year-after-year…”

10

WHAT OUR CUSTOMERS SAY

“The team from Durham have always been helpful and gave us confidence that arrangements for the security of the papers were taken seriously.

The move to CEM Entrance Assessments was an unqualified success. The papers set were fresh and stimulating, providing clear examples and thought-provoking tasks. The administration was straightforward, inspiring confidence (and gratitude!) in the staff responsible for supervising the whole process.

Most important of all for us, the assessments seemed to produce the right results: a cohort both eager and able to take on the challenge of high academic expectations.”

Tim Swain, Headmaster, Queen Mary’s Grammar School

“I have always been thoroughly impressed with CEM and would happily recommend them to any institution looking for support in designing an outstanding entrance test or curriculum free assessment”.

Mr S Norman, Headteacher, The Harvey Grammar School

“We have used CEM entrance assessments for three years now and have found their service to be efficient and helpful. There is always somebody available to respond to queries particularly in the context of dealing with appeals.

With regard to the outcomes in terms of the quality of our intake, there has been a marked improvement measurable through both MidYIS data and prior attainment at KS2. In school we have also noticed a difference in attitude and approach with students selected using CEM assessments.

The lack of availability of practice assessments is a significant help given the tutoring industry that is so prevalent. The administration of the assessments is very straightforward using the CD recordings for the instructions and staff at school have commented both on the ease of administration and the demanding nature of the questions.”

Mrs Alison Bruton, Headteacher, Queen Mary’s High School

11

The TeamCEM assessments are designed by a highly skilled development team from a variety of backgrounds, combining expertise from professionals across the education and academic sectors, with many years’ experience of developing robust assessments. The team works to create reliable and fair assessments, designed to enable the selection of candidates based on natural ability. The team are also heavily research active in areas surrounding test development and fair access to education.

Dr Sue Stothard, Head of Assessment DevelopmentSue is a psychologist with over 20 years of experience researching literacy and language development in children and developing literacy assessments. She has published many journal articles investigating the underlying cognitive skills associated with the typical and atypical development of reading comprehension, oral language and decoding skills. Current research focuses on the effects of demographic variables on test behaviour and performance.

Gideon Copestake, Statistical OfficerGideon has a background in primary education and worked on the PIPS project at CEM for 10 years. He was awarded his M.Sc. in Educational Assessment and is an expert in statistical methodologies.

John Ardley, Test DeveloperJohn has a degree in Computer Science and over 20 years of test development experience at CEM and has worked on a number of existing educational monitoring systems.

Emma Barthel, Test DeveloperEmma graduated with a Chemistry degree and worked as a primary school teacher for over 15 years before joining CEM as a test developer.

Lee Copping, Test DeveloperLee is a psychologist with a background in secondary education, interested in the use of assessment and monitoring systems in the education sector. His research interests focus on the effects of the environment on development.

Sarah Gott, Test DeveloperSarah has worked internationally and in the UK as a secondary teacher of mathematics for over 15 years. She is now involved in developing a range of assessment and monitoring systems across the primary and secondary education sectors.

Catherine McKenna, Assistant Test DeveloperCatherine is a psychologist with a particular interest in the assessment of non-verbal reasoning. She has previously worked on CEM’s secondary monitoring systems.

We pride ourselves on customer service, working with schools and Education authorities to understand their assessment needs. The operations team provide support and guidance for schools and authorities using our selection assessment services.

Neil Defty, Business Operations Manager

Hannah Fretwell, Operations Manager (Monday to Wednesday)

Richard O’Neill, Operations Manager (Thursday to Friday)

Sophie Dunn, Entrance Test Administrator

Find out moreIf you are interested in discussing things further please contact us:

Telephone: +44 (0) 191 334 4262

Email: [email protected]

Address: Entrance Assessments CEM, Rowan House Mountjoy Research Centre Durham University, Stockton Road Durham DH1 3UZ

For further information about CEM and the services we provide, please visit our website: www.cem.org

“Feedback can be provided in a variety of different ways…”

www.cem.org

selection assessment services test brochure.pdf · selection assessment services ... • all...

Documents