reasoning tests technical manual
Post on 10-Apr-2015
1.078 Views
Preview:
TRANSCRIPT
INTERNATIONAL
general,critical &graduateTEST BATTERYtechnical manual
ONTENTS1 THEORETICAL OVERVIEW
2 THE GRADUATE & GENERAL REASONING TESTS (GRT1 & GRT2)
3 THE CRITICAL REASONING TEST BATTERY (CRTB)
4 THE PSYCHOMETRIC PROPERTIES OF THE REASONING TESTS
5 ADMINISTRATION INSTRUCTIONS
6 REFERENCES
c
2
31 GRT2 STANDARDISATION SAMPLES
2 GENDER DIFFERENCES ON GRT23 CORRELATIONS BETWEEN GRT2 AND AGE
4 GENDER DIFFERENCES ON GRT15 RELATIONSHIP BETWEEN AGE AND GRT16 GENDER DIFFERENCES ON CRTB
7 RELATIONSHIP BETWEEN AGE AND CRTB
8 COEFFICIENT ALPHA FOR GRT2 SUB-SCALES
9 COEFFICIENT ALPHA FOR GRT1 SUB-SCALES
10 TEST-RETEST RELIABILITY ESTIMATES FOR GRT111 COEFFICIENT ALPHA FOR CRTB SUB-SCALES
12 DIFFERENCES BETWEEN COMPUTER VS CONVENTIONAL METHODS OF TEST
ADMINISTRATION
13 CORRELATIONS BETWEEN THE GRT1 SUB-SCALES
14 CORRELATIONS BETWEEN THE GRT2 SUB-SCALES
15 CORRELATIONS BETWEEN CRTB VERBAL & NUMERICAL
16 CORRELATIONS BETWEEN THE GRT1 AND AH5 SUB-SCALES
17 CORRELATIONS BETWEEN THE GRT2 AND AH3 SUB-SCALES
18 CORRELATIONS OF GRT2 WITH TECHNICAL TEST BATTERY
19 CORRELATIONS OF GRT2 WITH CLERICAL TEST BATTERY
20 CORRELATIONS BETWEEN THE EXPERIMENTAL VERSIONS OF THE VCR1 & NCR1AND THE VERBAL NUMERICAL SUB-SCALE OF THE AH5
21 CORRELATIONS BETWEEN VCR1 & NCR1 AND AH5 VERBAL/NUMERICAL
22 CORRELATIONS BETWEEN VCR1 & NCR1 AND WGCTA
23 CORRELATIONS BETWEEN VCR1 & NCR1 AND MAB (N=154)24 CORRELATIONS WITH GRT2 AND JOB PERFORMANCE IN AN INSURANCE SETTING
25 CORRELATIONS WITH GRT2 AND PERFORMANCE CRITERIA IN BANKING
26 CORRELATIONS BETWEEN GRT2 & SERVICE ENGINEER PERFORMANCE
27 CORRELATIONS BETWEEN GRT2 & PRINTER PERFORMANCE CRITERIA (N=70)28 CORRELATIONS BETWEEN GRT2 & PROFICIENCY CRITERIA
29 CORRELATIONS BETWEEN GRT2 & SUCCESSFUL APPLICANT FOR COMPONENT
COURSE
LIST OF TABLES
4
1THEORETICALOVERVIEWA major reason for using psychomet-
ric tests to aid selection decisions isthat they provide information thatcannot be obtained easily in other
ways. If such tests are not used thenwhat we know about the applicant is
limited to the information that canbe gleaned from an application formor CV, an interview and references. Ifwe wish to gain information about aperson’s specific aptitudes & abilities
and about their personality, atti-tudes and values then we have little
option but to use psychometric tests.In fact, psychometric tests can do
more than simply provide additionalinformation about the applicant.
They can add a degree of reliabilityand validity to the selection proce-dure that it is impossible to achievein any other way. How they do thisis best addressed by examining the
limitations of the informationobtained through interviews, appli-
cation forms and references andexploring how some of these limita-tions can be overcome by using psy-
chometric tests.
1 THE ROLE OF PSYCHOMETRIC TESTS
IN PERSONNEL SELECTION AND
ASSESSMENT
2 THE ORIGINS OF REASONING TESTS
6
THE ROLE OFPSYCHOMETRIC TESTS INPERSONNEL SELECTIONAND ASSESSMENTWhile much useful information canbe gained from the interview, whichclearly has an important role in anyselection procedure, it does nonethe-less suffer from a variety of weak-nesses. Perhaps the most importantof these is that the interview as beenshown to be a very unreliable way tojudge a person’s character. This isbecause it is an unstandardisedassessment procedure. That is to say,each interview will be different fromthe last. This is true even if the inter-viewer is attempting to ask the samequestions and act in the same waywith each applicant. It is preciselythis aspect of the interview that isboth its main strength and its mainweakness. The interview enables usto probe each applicant in depth anddiscover individual strengths andweaknesses. Unfortunately, the inter-views unstandardised, idiosyncraticnature makes it difficult to compareapplicants, as it provides no base lineagainst which to contrast intervie-wees’ differing performances. Inaddition, it is likely that differentinterviewers may come to radicallydifferent conclusions about the sameapplicant. Applicants will responddifferently to different interviewers,quite often saying very differentthings to them. In addition, what anyone applicant might say will beinterpreted quite differently by eachinterviewer. In such cases we have toask which interviewer has formedthe correct impression of the candi-date? This is a question to whichthere is no simple answer.
A further limitation of the inter-view is that it only assesses thecandidate’s behaviour in one setting,and with regard to a small numberof people. How the candidate mightact in different situations and withdifferent people (e.g. when dealingwith people on the shop floor) is notassessed, and cannot be predictedfrom an applicant’s interview perfor-mance. Moreover, the interviewprovides no reliable informationabout the candidate’s aptitudes andabilities. The most we can do is askthe candidate about his strengthsand weaknesses, a procedure thathas obvious limitations. Thus therange, and reliability of the informa-tion that can be gained through aninterview are limited.
There are similar limitations onthe range and usefulness of the infor-mation that can be gained fromapplication forms or CV’s. Whilework experience and qualificationsmay be prerequisites for certainoccupations, in and of themselvesthey do not determine whether aperson is likely to perform well orbadly. Experience and academicachievement is not always a goodpredictor of ability or future success.While such information is importantit may not be sufficient on its own toenable us to confidently choosebetween applicants. Thus aptitudeand ability tests are likely to play asignificant role in the selectionprocess as they provide informationon a person’s potential and not justtheir achievements to date.
7Moreover, application forms tell uslittle about a person’s character. It isoften a candidate’s personality thatwill make the difference between anaverage and an outstanding perfor-mance. This is particularly truewhen candidates have relativelysimilar records of achievement andpast performance. Therefore, person-ality tests can play a major role inassisting selection decisions.
References do provide some usefulinformation but mainly for verifica-tion purposes. While past perfor-mance is undoubtedly a good predic-tor of future performance referencesare often not good predictors of pastperformance. If the name of thereferee is supplied by the applicant,then it is likely that they have chosensomeone they expect to speak highlyof them. They will probably haveavoided supplying the names ofthose who may have a less positiveview of their abilities. Aptitude andability tests, on the other hand, giveus an indication of the applicant’sprobable performance under examconditions. This is likely to be a truereflection of the person’s ability.
What advantages do psychometrictests have over other forms of assess-ment? The first advantage they haveis that they add a degree of reliabilityto the selection procedure thatcannot be achieved without their use.Test results can be representednumerically making it easy both tocompare applicants with each other,and with pre-defined groups (e.g.successful vs. unsuccessful job
incumbents). In the case of personal-ity tests the test addresses the issueof how the person characteristicallybehaves in a wide range of differentsituations and with different people.Thus psychometric tests of personal-ity, aptitude and ability provide arange of information that are noteasily and reliably assessed in otherways. Such information can fillimportant gaps which have not beenassessed by application forms, inter-views and references. It can also raisequestions that can later be directlyaddressed in the interview. It is forthis reason that psychometric testsare being used increasingly inpersonnel selection. Their use adds adegree of breadth to assessment deci-sions which cannot be achieved inany other way.
8THE ORIGINS OFREASONING TESTSThe assessment of intelligence orreasoning ability is perhaps one ofthe oldest areas of research interestin psychology. Gould (1981) hastraced attempts to scientificallymeasure psychological aptitudes andabilities to the work of Galton at theend of the last century. Prior toGalton’s pioneering work, however,interest in this area was aroused byphrenologists’ attempts to assessmental ability by measuring the sizeof people’s heads. Reasoning tests, intheir present form, were first devel-oped by Binet, a French educational-ist who published the first test ofmental ability in 1905.
Binet was concerned with assess-ing the intellectual development ofchildren and to this end invented theconcept of mental age. Questions,assessing academic ability, weregraded in order of difficulty accord-ing to the average age at which chil-dren could successfully completeeach item. From the child’s perfor-mance on such a test it was possibleto derive its mental age. Thisinvolved comparing the performanceof the child with the performance ofthe ‘average child’ from different agegroups. If the child performed at thelevel of the average 10 year old, thenthe child was said to have a mental
age of 10, regardless of its chrono-logical age. From this idea theconcept of the Intelligence Quotient(IQ) was developed by William Stern(1912) who defined it as mental agedivided by chronological age multi-plied by 100. Previous to Stern’spaper chronological age had beensubtracted from mental age toprovide a measure of mental alert-ness. Stern showed that it was moreappropriate to take the ratio of thesetwo constructs, which would providea measure of the child’s intellectualdevelopment relative to other chil-dren. He further proposed that thisratio should be multiplied by 100 forease of interpretation; thus avoidingcumbersome decimals.
Binet’s early tests were subse-quently revised by Terman et al.(1917) to produce the now famousStanford-Binet IQ test. These earlyIQ tests were first used for selectionby the American’s during the firstworld war, when Yerkes (1921)tested 1.75 million soldiers with thearmy alpha and beta tests. Thus bythe end of the war, the assessment ofreasoning ability had firmly estab-lished its place within psychology.
2THE GRADUATE& GENERALREASONING TESTS(GRT1 & GRT2)
bkResearch in the area of intelligencetesting has consistently demon-strated that three aptitude domains:Verbal, Numerical and AbstractReasoning Ability (Heim, 1970).Consequently, the GRT1 and GRT2have been designed to measure justthese three areas of ability. Verbaland Numerical ability assess, as theirrespective names would suggest, theability to use words and numbers ina rational way, correctly identifyinglogical relationships between theseentities and drawing conclusions andinferences from them. Abstract,reasoning assesses the ability to iden-tify logical relationships betweenabstract spatial relationships andgeometric patterns. Many psycholo-gists argue that Abstract Reasoningtests assess the purest form of ‘intel-ligence’. That is to say, these tests arethe least affected by educationalexperience and assess what somemight term ‘innate’ reasoning ability.Namely, the ability to solve abstract,logical problems which require noprior knowledge or educationalexperience.
Research has clearly demonstratedthat in order to accurately assessreasoning ability it is necessary todevelop tests which have beenspecifically designed to measure thatability in the population underconsideration. That is to say, weneed to be sure that the test has beendeveloped for use on the particulargroup being tested, and thus that itis appropriate for that particulargroup. There are two ways in whichthis is important. Firstly, it is impor-tant that the test has been developedin the country in which it is intendedto be used. This ensures that theitems in the test are drawn from acommon, shared cultural experience,giving each candidate an equalopportunity to understand the logicwhich underlies each item. Secondly,it is important that the test isdesigned for the particular abilityrange on which it is to be used. Atest designed for those of averageability will not accurately distinguishbetween people of high ability as alltheir scores will cluster towards thetop end of the scale. Similarly, a testdesigned for people of high abilitywill be of little practical use if givento people of average ability. Not onlywill the test not discriminate betweenapplicants, as all the scores willcluster towards the bottom of thescale, but also as the questions willbe too difficult for most of the appli-cants they are likely to lose motiva-tion, producing artificially lowscores. For this reason two versionsof this reasoning test were developed.One developed for the general popu-lation (the average ability range) andone for the graduate population.
blIn constructing the items in the
GRT1 and GRT2 which measurethese reasoning abilities a number ofguide lines were borne in mind.Firstly, and perhaps most impor-tantly, each item was constructed sothat only a minimal educational levelwas needed in order to be able tocorrectly solve each item. Thus wetried to ensure that each item was ameasure of ‘reasoning ability’, ratherthan being a measure of specificknowledge or experience. Forexample, in the case of the numericalitems the calculations involved insolving each item are relativelysimple, with the difficulty of the itembeing due to the logic which under-lies that question rather than beingdue to the need to use complexmathematical operations to solvethat item. (It should be notedhowever that in the case of the GRT1a higher level of education wasassumed, as the test was designed fora graduate population). Secondly, anumber of different item types (e.g.odd one out, word meanings etc.)were used to measure each aspect ofreasoning ability. This was done inorder to ensure that each sub-scalemeasures a broad aspect of reasoningability (e.g. Verbal ReasoningAbility), rather than measuring avery specific aptitude (e.g. vocabu-lary). In addition, the use of differentitem types ensures that the test ismeasuring different components ofreasoning ability. For example theability to understand analogies,inclusion/exclusion criteria for classmembership etc.
bm
3THE CRITICALREASONING TESTBATTERY (CRTB)
boResearch has clearly demonstratedthat in order to accurately assessreasoning ability it is necessary todevelop tests which have beenspecifically designed to measure thatability in the population underconsideration. That is to say, weneed to be sure that the test has beendeveloped for use on the particulargroup being tested, and thus isappropriate for that particulargroup. There are two ways in whichthis is important. Firstly, it is impor-tant that the test has been developedin the country in which it is intendedto be used. This ensures that theitems in the test are drawn from acommon, shared cultural experience,giving each candidate an equalopportunity to understand the logicwhich underlies each item. Secondly,it is important that the test isdesigned for the particular abilityrange on which it is to be used. Atest designed for those of averageability will not accurately distinguishbetween people of high ability as allthe scores will cluster towards thetop end of the scale. Similarly, a testdesigned for people of high abilitywill be of little use if given to peopleof average ability. Not only will it notdiscriminate between applicants, asall the scores will cluster towards thebottom of the scale, but also as thequestions will be too difficult formost of the applicants they are likelyto be de-motivated, producing artifi-cially low scores. Consequently, the
VCR1 and NCR1 have been devel-oped on data from undergraduates.That is to say, people of aboveaverage intelligence, who are likelyto find themselves in senior manage-ment positions as their career devel-ops.
In constructing the items in theVCR1 and NCR1 a number of guidelines were borne in mind. Firstly,and perhaps most importantly,special care was taken when writingthe items to ensure that in order tocorrectly solve each item it wasnecessary to draw logical conclusionsand inferences from the stempassage/table. This was done toensure that the test was assessingcritical (logical/deductive) reasoningrather than simple verbal/numericalchecking ability. That is to say, theitems assess a person’s ability tothink in a rational, critical way andmake logical inferences from verbaland numerical information, ratherthan simply check for factual errorsand inconsistencies.
In order to achieve this goal forthe Verbal Critical Reasoning(VCR1) test two further points wereborn in mind when constructing thestem passages for the VCR1. Firstly,the passages were kept fairly shortand cumbersome grammaticalconstructions were avoided, so that aperson’s scores on the test would notbe too affected by reading speed;thus providing a purer measure ofcritical reasoning ability. Secondly,
bp
care was taken to make sure that thepassages did not contain any infor-mation which was counter-intuitive,and was thus likely to create confu-sion. To increase the acceptability ofthe test to applicants the themes ofthe stem passages were chosen to berelevant to a wide range of businesssituations. As a consequence of theseconstraints the final stem passageswere similar in many ways to theshort articles found in the financialpages of a daily newspaper.
Finally an extended responseformat was chosen for the VCR1.While many critical reasoning testsonly ask the applicant to make thedistinction between whether thetarget statement is true, false orcannot be inferred from the stempassage the response format for theVCR1 was extended to include thejudgement of whether the targetstatement was probably or definitelytrue/false given the information inthe passage. This was done in orderto decrease the chance of guessing acorrect answer from 33% to 25%.With guessing having substantiallyless impact on a candidate’s finalscore, it was thus possible to decreasethe number of items in the test thatwere needed for it to be reliable.
bq
4THE PSYCHOMETRICPROPERTIES OF THEREASONING TESTSThis chapter will present details con-
cerning the psychometric propertiesof the reasoning Tests. The aim will
be to show that these measures fulfilvarious technical requirements, in
the areas of standardisation, reliabil-ity and validity, which ensure the
psychometric soundness of the test.
1 INTRODUCTION
2 STANDARDISATION
3 RELIABILITY OF THE REASONING
TESTS
4 METHOD EFFECTS
5 VALIDITY
6 THE STRUCTURE OF THE REASONING
TESTS
7 THE CONSTRUCT VALIDITY OF THE
REASONING TEST
8 CRITERION-RELATED VALIDITY
bs INTRODUCTION
Standardisation : NormativeNormative data allows us to comparean individuals score on a standard-ised scale against the typical scoreobtained from a clearly identifiable,homogeneous group of people.
RELIABILITY
The property of a measurementwhich assesses the extent to whichvariation in measurement is due totrue differences between people onthe trait being measure or tomeasurement error.
In order to provide meaningfulinterpretations, the reasoning testswere standardised against a numberof relevant groups. The constituentsamples are fully described in thenext section. Standardisation ensuresthat the measurements obtainedfrom a test can be meaningfullyinterpreted in the context of a rele-vant distribution of scores. Anotherimportant technical requirement fora psychometrically sound test is thatthe measurements obtained fromthat test should be reliable.
Reliability is generally assessedusing two specific measures, onerelated to the stability of scale scoresover time, the other concerned withthe internal consistency, or homo-geneity of the constituent items thatform a scale score.
Reliability : StabilityAlso known as test-retest reliability,an assessment is made of the similar-ity of scores on a particular scaleover two or more test occasions. Theoccasions may be from a few hours,days, months or years apart.
Normally Pearson correlation coeffi-cients are used to quantify the simi-larity between the scale scores overthe two or more occasions.
Stability coefficients provide animportant indicator of a test’s likelyusefulness of measurement. If thesecoefficients are low (< approx. 0.6)then it is suggestive of either that theabilities/behaviours/attitudes beingmeasured are volatile or situationallyspecific, or that over the duration ofthe retest interval, situational eventshave made the content of the scaleirrelevant or obsolete. Of course, theduration of the retest intervalprovides some clue as to which effectmay be causing the unreliability ofmeasurement. However, the secondmeasure of a scales reliability alsoprovides valuable information as towhy a scale may have a low stabilitycoefficient. The most commonmeasure of internal consistency isCronbach’s Alpha. If the items on ascale have high inter-correlationswith each other, and with the totalscale score, then coefficient alphawill be high. Thus a high coefficientalpha indicates that the items on thescale are measuring very much thesame thing, while a low alpha wouldbe suggestive of either scale itemsmeasuring different attributes or thepresence of error.
Reliability : InternalConsistencyAlso known as scale homogeneity, anassessment is made of the ability ofthe items in a scale to measure thesame construct or trait. That is aparameter can be computed that
bt
indexes how well the items in a scalecontribute to the overall measure-ment denoted by the scale score. Ascale is said to be internally consis-tent if all the constituent itemresponses are shown to be positivelyassociated with their scale score.
The fact that a test has high inter-nal consistency & stability coeffi-cients only guarantees that it ismeasuring something consistently. Itprovides no guarantee that the test isactually measuring what it purportsto measure, nor that the test willprove useful in a particular situation.Questions concerning what a testactually measures and its relevancein a particular situation are dealtwith by looking at the tests validity.Reliability is generally investigatedbefore validity as the reliability oftest places an upper limit on testsvalidity. It can be mathematicallydemonstrated that a validity coeffi-cient for a particular test can notexceed that tests reliability coeffi-cient.
VALIDITY
The ability of a scale score to reflectwhat that scale is intended tomeasure. Kline’s (1993) definition is“A test is said to be valid if itmeasures what it claims to measure”.
Validation studies of a test investi-gate the soundness and relevance ofa proposed interpretation of thattest. Two key areas of validation areknown as criterion validity andconstruct validity.
Validity : Criterion ValidityCriterion validity involves translat-ing a score on a particular test into aprediction concerning what could beexpected if another variable wasobserved.
The criterion validity of a test isprovided by demonstrating thatscores on the test relate in somemeaningful way with an externalcriterion. Criterion validity comes intwo forms – predictive & concurrent.Predictive validity assesses whether atest is capable of predicting anagreed criterion which will be avail-able at some future time – e.g. can atest predict the likelihood of someonesuccessfully completing a trainingcourse. Concurrent validity assesseswhether the scores on a test can beused to predict a criterion measurewhich is available at the time of thetest – e.g. can a test predict currentjob performance.
Validity : Construct ValidityConstruct validity assesses whetherthe characteristic which a test isactually measuring is psychologicallymeaningful and consistent with thetests definition.
The construct validity of a test isassessed by demonstrating that thescores from the test are consistentwith those from other major testswhich measure similar constructsand are dissimilar to scores on testswhich measure different constructs.
ck STANDARDISATION
For each of the three reasoningbatteries, information is provided onthe constituent norm samples, ageand gender differences where theyapply. All normative data is availablefrom within the GeneSys systemwhich computes for any given rawscore, the appropriate standardisedscores for the selected referencegroup. In addition the GeneSys™software allows users to establishtheir own in-house norms to allowmore focused comparison withprofiles of specific groups.
GRT2 NORMATIVE DATA
The total norm base of the GRT2 isbased on a general population normas well as a number of morespecialised norm groups. Theseinclude undergraduates, technicalstaff, personnel managers, customerservice staff, management applicantsetc. detailed in Table 1.
GRT2 GENDER AND AGEDIFFERENCES
Gender differences on GRT2 wereexamined by comparing results ofalmost equal numbers of males andfemale respondents matched as faras possible for educational and socio-economic status. Table 2 providesmean scores for males and femaleson each of the GRT2 scales as well asthe t-value for mean score differ-ences.
The results below demonstrategender differences on NR2 with themale mean score just over two rawscore points higher than that of thefemales. This is in line with othernumerical measures. More surprisingperhaps, is that no differences wereobserved on either the Verbal andAbstract measures.
The effect of age on GRT2 scoreswas examined using a sample of1441 respondents on whom age datawas available (see Table 3). Whereasthere is a negative relationship withGRT2 scores and age for all the threemeasures, this tendency is highlysignificant on the Abstract (AR2),suggesting, in line with expectationsthat fluid ability may be more likelyto decline with age than crystallisedability.
cl
General Population
Telesales Applicants
HE College Students
Customer Service Clerks
Technical Staff
Financial Consultants
HR Professionals
Service Engineers
Males N
3177
175
5
28
122
58
30
86
Females N
1236
391
158
87
24
20
38
8
Mean Age
35.10
27.16
16.25
26.46
29.38
32.94
36.31
26.56
Range
16-63
16-55
15-41
19-50
16-58
20-53
22-55
18-58
SD Age
8.87
8.29
3.14
7.19
9.92
9.20
7.94
6.87
Table 1: GRT2 Standardisation Samples
GRT2
VR2
NR2
AR2
MeanFemales
23.36
15.77
16.95
MeanMales
23.22
17.91
17.40
t-value
.3404
-5.424
-1.259
df
1438
1438
1438
p
.734
.000
.208
FemalesN
852
852
852
MalesN
588
588
588
Table 2: Gender differences on GRT2
GRT2
VR2
NR2
AR2
AGE
.11
.10
-.37
Table 3: Pearson correlations between GRT2 and age
cmGRT1 GENDER AND AGEDIFFERENCES
Gender differences on GRT1 wereexamined by comparing results ofalmost equal numbers of males andfemale respondents matched as faras possible for educational and socio-economic status. Table 4 providesmean scores for males and femaleson each of the sub-scales of theGRT1 as well as the t-value for meanscore differences.
Consistent with findings on theGRT2, the only significant scoredifference is registered on theNumerical (NR1) with males regis-tering a higher mean score. Nosignificant differences were observedfor either the Verbal or the Abstractcomponents of the GRT1.
The effect of age on GRT1 scoreswas examined using a sample of 499respondents on whom age data wasavailable (see Table 5). Whereas theVerbal and Numerical were found tobe unrelated to age, the Abstractshowed a significant negative rela-tionship, consistent with expecta-tions. The correlations are lowerthan those obtained with the GRT2,although this may be explained bythe sample which in this case is morerestricted in the range of observedscores.
CRTB GENDER AND AGEDIFFERENCES
Gender differences on CRTB wereexamined by comparing results ofmales and female respondentsmatched as far as possible for educa-tional and socio-economic status.Table 6 opposite provides meanscores for males and females on boththe verbal and numerical sub-scaleof the CRTB scales as well as the t-value for mean score differences.
While female respondents registermarginally higher verbal reasoningscores (VCR1) than males, this is notstatistically significant. A significantgender difference was observed onthe Numerical (NCR1), with malesregistering over four raw score pointshigher mean score. This is consistentwith other measures of numericalability.
The effect of age on CRTB scoreswas examined using a sample of 359respondents on whom age data wasavailable (see Tabel 7). Unlike theGeneral and Graduate Reasoningtests, the correlations with age arepositive for the Critical Reasoning,although only marginal and non-significant for the Verbal. Thissuggests that the Numerical at least,may be measuring more of anacquired ability, which is more posi-tively influenced by experience thanother more classic measures ofnumerical ability.
cn
GRT1
VR1
NR1
AR1
MeanFemales
16.73
12.98
14.11
MeanMales
16.61
14.91
14.27
t-value
.260
-4.255
-.383
df
497
497
497
p
.795
.000
.702
FemalesN
251
251
251
Males N
248
248
248
Table 4: Gender Differences on GRT1
CRTB
VCR1
NCR1
F
18.34
10.19
M
17.78
14.40
t-value
.501
-3.578
df
134
134
p
.6178
.0004
F
47
47
M
89
89
Table 6: Gender differences on CRTB
GRT1
VR1
NR1
AR1
AGE
-.00
-.05
-.27
Table 5: Relationship between Age and GRT1
CRTB
VRC1
NRC1
AGE
.03
.18
Table 7: Relationship between Age and CRTB
co
If a reasoning test is to be used forselection and assessment purposesthe test needs to measure each of theaptitude or ability dimensions it isattempting to measure reliably, forthe given population (e.g. graduateentrants, senior managers etc.). Thatis to say, the test needs to be consis-tently measuring each ability so thatif the test were to be used repeatedlyon the same candidate it wouldproduce similar results.
It is generally recognised thatreasoning tests are more reliablethan personality tests and for thisreason high standards of reliabilityare usually expected from such tests.While many personality tests areconsidered to have acceptable levelsof reliability if they have reliabilitycoefficients in excess of .7, reasoningtests should have reliability coeffi-cients in excess of .8.
GRT2 INTERNAL CONSISTENCY
Table 8 presents alpha coefficientsfor the three sub-scales of the GRT2(n=135). Each of these reliabilitycoefficients is substantially greaterthan .8, clearly demonstrating thatgeneral population version of theGRT is highly reliable.
GRT1 INTERNAL CONSISTENCY
Table 9 presents alpha coefficientsfor the three sub-scales of the GRT1(n=109). Each of these reliabilitycoefficients is greater than .8, clearlydemonstrating that the graduateversion of the GRT meets acceptablelevels of internal consistency.
GRT1 TEST-RETEST
A sample of 70 undergraduatestudents completed the GRT1 on twoseparate occasions with a four weekinterval. Table 10 provides uncor-rected correlations for each measure.
Although the Abstract falls some-what below the ideal, the test-retestcorrelations are generally of a highand acceptable level which, inconjunction with the internal consis-tency data would demonstrate thatthe GRT1 is a reliable measure ofgeneral reasoning ability.
Table 11 presents alpha coeffi-cients for the two sub-scales of theCritical Reasoning Test. Each ofthese reliability coefficients is .8 orgreater, clearly demonstrating thatCritical Reasoning Tests reachacceptable levels of reliability.
RELIABILITY OF THEREASONING TESTS
cp
GRT2
Verbal (VR2)
Numerical (NR2)
Abstract (AR2)
r
.83
.84
.83
Table 8: Coefficient Alpha for GRT2Sub-scales (n = 135)
GRT1
Verbal (VR1)
Numerical (NR1)
Abstract (AR1)
r
.82
.85
.84
Table 9: Coefficient Alpha for GRT1Sub-scales (n = 109)
CRTB
Verbal Critical Reasoning (VCR1)
Numerical Critical Reasoning (NCR1)
r
.80
.82
Table 11: Coefficient Alpha for CRTB Sub-scales (N=134)
GRT1
VR1
NR1
AR1
VR1
.79
.35
.22
NR1
.31
.78
.38
AR1
.09
.39
.74
Table 10: Test-retest reliability esti-mates for GRT1 (N=70)
cq METHOD EFFECTS
One important aspect of the changefrom paper and pencil to computer-based administration, concerns theeffects the change in test formatmight have on the available norma-tive data for a test. In other wordsdoes the translation of a test tocomputer format alter the nature ofthe test itself? It is possible that thescore range of a test administered inquestion booklet form may be differ-ent from the range when the test isadministered via a computer. Giventhat most of the normative dataavailable for mainstream psychomet-ric tests was collected from paperand pencil test administration thequestion is far from academic. Forinstance, many organisations willadminister computer-based tests attheir head office location but will usepaper and pencil format when theirtesters visit remote locations. Ifcomparison is to be made across agroup in which some individualsreceived paper and pencil testing andsome computer testing then theimportance of the question ofnormative comparability can not beoverstated.
Very little research attention hasbeen paid to this topic, which issurprising given the possible impactthat format differences would have.Roid (1986) has indicated that whatlittle evidence is available concerningpaper and pencil v. computerformats suggests that computeradministration of most tests does notchange the score range enough toaffect the normative basis of the test.While this is somewhat reassuringthe number of studies looked at by
Roid was small and consistedentirely of American investigations.
As publishers of a wide range oftests which can be administered bothby paper and pencil and computer,Psytech International recentlyconducted a study looking at theeffect of administration format onreasoning test performance. The Testchosen was the Graduate ReasoningTest, a graduate level test comprisingthree sub-tests – verbal, numericaland abstract. This test was chosen asbeing representative of many main-stream reasoning tests. The GraduateReasoning Test also gives an oppor-tunity to test whether the representa-tional format of a test is important.It could be the case that computerpresentation of graphical material,such as is found in mechanical andspatial reasoning tests, might lead toperformance differences while alpha-numeric material does not. With theGRT1 the abstract subtest uses agraphical format while the verbaland numerical sub-scales use alpha-numeric formats.
cr
THE STUDY
A group of 80 university undergrad-uates took part in this investigation.The students were divided into fourgroups. Each group completed theGraduate Reasoning Test twice withan interval of two weeks betweensuccessive administrations. Two ofthe groups completed the sameformat of the test on each occasion –i.e. paper/paper orcomputer/computer. The other twogroups experienced both formats ofthe test, each group in a differentorder – i.e. paper/computer orcomputer/paper.
RESULTS
An independent t-test was used totest whether any differences existedbetween the mean scores on firstadministration for those studentswho completed the paper version ofthe GRT1 and those receiving theGeneSys Administered version. Ascan be seen from Table 12 a signifi-cant difference was found for boththe Numerical and Abstract sub-scales in that students who hadreceived the computer administeredversion of the test tended to scoresignificantly higher than those whohad received the paper version first.Table 12 provides similar data forthe two test formats on secondadministration. It can be seen fromthis table that no significant differ-ences existed between the differentformats for second administration.
CONCLUSION
The results of this study show that asfar as the Graduate Reasoning Test,at least, is concerned the format inwhich the test is administered doesaffect, to some extent, the scoresobtained. No differences between thegroup means were detected for eitherof the three sub-tests of the GraduateReasoning Test. It was also the casethat no difference was foundbetween the graphical representationof the abstract sub-test and thealpha-numerical representation ofthe verbal and numerical sub-tests.Furthermore no interaction effectswere observed which indicates thatthe well established phenomenon of‘practice effects’ does not differ withthe nature of the test medium.
These results provide some confi-dence that the changeover frompaper and pencil to computer-basedtesting will not require the re-stan-dardisation of tests. It would seemfrom this study that administrationof ability tests by either computer orpaper and pencil will produce similarperformance levels. Thus it isperfectly acceptable to compare indi-viduals who were tested using differ-ent formats.
GRT1
VR1
NR1
AR1
Mean
PC
19.09
14.53
15.58
Mean
Paper
18.48
13.86
14.79
t-value
.876
.927
1.117
df
158
158
158
p
.382
.355
.266
N-PC
80
80
80
N-Paper
80
80
80
SD-PC
4.26
3.97
3.81
SD-
Paper
4.58
5.01
5.03
F-ratio
variance
1.15
1.59
1.74
P
variance
.532
.0419
.0146
Table 12: Differences between Computer vs. Conventional Methods of Test Administration
cs VALIDITY
Whereas reliability assess the degreeof measurement error of a reasoningtest, that is to say the extent to whichthe test is consistently measuring oneunderling ability or aptitude, validityaddresses the question of whether ornot the scale is measuring the char-acteristic it was developed tomeasure. This is clearly of keyimportance when using a reasoningtest for assessment and selectionpurposes. In order for the test to be auseful aid to selection we need to
know that the results are reliable andthat the test is measuring the apti-tude it is supposed to be measuring.Thus after we have examined a test’sreliability we need to address theissue of validity. We traditionallyexamine the reliability of a testbefore we explore its validity as reli-ability sets the lower bound of ascale’s validity. That is to say a testcannot be more valid than it is reli-able.
GRT1 sub-scale
Verbal (VR1)
Numerical (NR1)
Abstract (AR1)
VR1
_
NR1
.42
_
AR1
.30
.55
_
Table 13: Product-moment Correlationsbetween the GRT1 Sub-scales (n=499)
GRT2 sub-scale
Verbal (VR2)
Numerical (NR2)
Abstract (AR2)
VR2
_
NR2
.60
_
AR2
.56
.65
_
Table 14: Product-moment Correlationsbetween the GRT2 Sub-scales (n = 1441)
ctTHE STRUCTURE OF THEREASONING TESTS
Specifically we are concerned thatthe test’s sub-scales are correlatedwith each other in a meaningful way.For example, we would expect thedifferent sub-scales of a reasoningtest to be moderately correlated aseach will be measuring a differentfacet of general reasoning ability.Thus if such sub-scales are not corre-lated with each other we mightwonder whether each is a goodmeasure of reasoning ability.Moreover, we would expect differentfacets of verbal reasoning ability(e.g. vocabulary, similarities etc.) tobe more highly correlated with eachother than they are with a measureof numerical reasoning ability.Consequently, the first way in whichwe might assess the validity of areasoning test is by exploring therelationship between the test’s sub-scales.
THE GRADUATE REASONINGTESTS (GRT1)
Table 13, which presents PearsonProduct-moment correlationsbetween the three sub-scales of theGRT1 demonstrates two things.Firstly, the relatively strong correla-tions between each of the sub-scalesindicate that each is measuring onefacet of an underlying ability. This isclearly consistent with the design ofthis test, where each sub-scale wasintended to assess a different facet ofreasoning ability or mental alertness.Secondly, the fact that each sub-scaleaccounts for less than 30% (r < .55)of the variance in the other sub-scales indicates that the Verbal,Numerical and Abstract Reasoningsub-scales of the GRT1 are measur-ing different facets of reasoningability, as they were designed to.Moreover, this is what we would infact predict from research in the areaintelligence testing (Heim, 1970).
THE GENERAL REASONINGTESTS (GRT2)
Table 14, which presents PearsonProduct-moment correlationsbetween the three sub-scales of theGRT2 demonstrates two things.Firstly, the relatively strong correla-tions between each of the sub-scalesindicate that each is measuring oneunderlying characteristic, which inthis case we might assume to bereasoning ability or mental alertness.Thus these relatively strong correla-tions between the sub-scales areconsistent with our intention toconstruct a test which measuresgeneral reasoning ability. Secondly,the fact that each sub-scale accountsfor less than 45% (r < .65) of thevariance in the other sub-scales indi-cates that the Verbal, Numerical andAbstract Reasoning sub-scales of theGRT2 are still measuring distinctaspects of reasoning ability.
THE CRITICAL REASONINGTESTS (CRTB)
Table 15, which presents the PearsonProduct moment correlation betweenthe two sub-tests of the CRTB,demonstrates that while the Verbaland Numerical sub-tests are margin-ally correlated, they neverthelessmeasuring quite distinct abilities,sharing only 10% of common vari-ance.
CRTB
Verbal
Numerical
Verbal
1
.352
Numerical
.352
1
Table 15: Product-moment Correlationsbetween CRTB Verbal & Numerical (n=352)
dkTHE CONSTRUCT VALIDITYOF THE REASONING TESTS
As an evaluation of construct valid-ity, the Psytech Reasoning Tests wereadministered with other widely usedmeasures of similar constructs.
In the case of the General andGraduate reasoning tests, the AHseries was considered to be a suitableexternal measure. The AH Series oftests is one of the most widelyrespected range of reasoning testswhich have been developed on aU.K. population. Within this seriesthere are tests which have beenspecifically designed for use on boththe general population (AH2/ AH3/AH4) and the graduate population(AH5/AH6). Developed by AliceHeim (1968, 1974) of CambridgeUniversity the AH series has becomesomething of a benchmark againstwhich to compare the performanceof other reasoning tests.
As the original Critical ThinkingAppraisal, The Watson-Glaser (W-GCTA) (Watson & Glaser (1991) hasset the standard in the assessment ofabilities that are of relevance inmanagement decision-making.
Thus we chose to explore theconstruct validity of the GRT1 andthe GRT2 by comparing their perfor-mance against that of the AH3 andAH5 respectively, while the constructvalidity of the CRTB is examined bycomparing its performance againstboth the AH5 and the Watson-GlaserCritical Thinking Appraisal.
THE RELATIONSHIP BETWEENTHE GRT1 AND AH5
Table 16 presents product-momentcorrelations between the sub-scalesand the total scale scores of the AH5and the GRT1. The AH5, which hasbeen developed for use on a graduatepopulation, has a Verbal/Numericalsub-scale which combines verbal andnumerical items and an Abstract, orDiagrammatic, reasoning sub-scale.These along with the total scale scoreon the AH5 (the sum of the sub-scales) were correlated with theGRT1 sub-scale scores and the totalscale score. The correlations with thetotal scale scores were includedwithin this table, even though thetotal scale scores are simple compos-ites of the sub-scale scores, as theyprovide a measure of general (g),rather than specific (e.g. verbal,numerical etc.) mental aptitudes.
Table 16 provides clear supportfor the concurrent validity of theGRT1 against the AH5. The correla-tions between each of the GRT1 sub-scales and their comparable AH5sub-scales are high, indicating thatthey are measuring similarconstructs. In addition, this tableprovides some evidence in support ofthe discriminant validity of thesesub-scales. That is to say, each of theGRT1 sub-scales is more highlycorrelated with its comparable sub-scale on the AH5 than it is with theAH5 sub-scale measuring a differentspecific mental ability. For example,while the VR1 has a correlation of.69 with the Verbal/Numerical sub-
dl
scale of the AH5, its correlation withthe Abstract sub-scale is only .35. Inaddition, the extremely high correla-tion (r=.84) between the total scalescores of the AH5 and the GRT1 indi-cates that, as a whole, this battery oftests is a good measure of generalreasoning ability (g).
THE RELATIONSHIP BETWEENTHE GRT2 AND AH3
Table 17 presents Pearson Product-moment correlations (n=81) betweeneach of the GRT2 sub-scales with eachof the sub-scale scores of the AH3 andthe total scale score. In addition to thehigh correlations between these sub-scales it is also worth noting theextremely high correlation (r=.82)between the total scale scores on thesetwo tests. These results clearly demon-strate that the GRT2 is measuring thetrait of general reasoning ability whichis assessed by the AH3. We shouldhowever note that the correlationsbetween the sub-scales provide noclear support for the discriminantvalidity of the GRT2. That is to say,the correlations between each of theGRT2 sub-scales and their respectiveAH3 sub-scales (e.g. the VR2 with theAH3 Verbal) are not significantlyhigher than are the correlations acrosssub-scales (e.g. the VR2 with the AH3Numerical and Verbal).
SUB-SCALE
AH5 Verbal/Numerical
AH5 Abstract
AH5 Total
VR1
.69
.51
.69
NR1
.70
.67
.79
AR1
.35
.72
.65
TOTAL
.70
.74
.84
Table 16: Product-moment correlations between the GRT1 andAH5 Sub-scales
SUB-SCALE
AH5 Verbal
AH3 Numerical
AH5 Perceptual
AH5 Total
VR2
.63
.58
.54
.70
NR2
.63
.76
.55
.78
AR2
.61
.76
.56
.78
GRTTotal
.73
.70
.76
.82
Table 17: Product-moment Correlations between theGRT2 and AH3 Sub-scales
dm
RELATIONSHIP WITH GRT2AND OTHER MEASURES
As a part of a number of data-collec-tion exercises, the GRT2 was appliedwith a number of alternativemeasures, namely the technical TestBattery (TTB2) and Clerical TestBattery (CTB2).
Technical Test Battery (TTB2)A sample of 94 trainee Mechanicalapprentices completed both theGRT2 and the Technical Test Batteryas part of a validation exercise. TheGRT2 sub-scales register modestcorrelations with the componentsmeasures of the Technical TestBattery, although this is no morethan would be expected from differ-ent aptitude measures with noneexceeding .50 (see Table 18).
Clerical Test Battery (CTB2)A sample of 54 clerical staff workingfor a major bank completed theVerbal reasoning Test (VR2) as partof an assessment of Clerical apti-tudes which included components ofthe Clerical Test Battery (CTB2).
The strongest observed correlationwas not with the Spelling measure(SP2) as expected but with OfficeArithmetic (NA2). Examination ofNA2 does reveal a fairly high verbalproblem-solving element, which mayexplain this. The Clerical CheckingTest (CC2) only registered a modestcorrelation which is as expected of ameasure which relies only to alimited extent on general ability (seeTable 19).
THE RELATIONSHIP BETWEENTHE CRTB AND AH5
In Tables 20 and 21 we present twosets of data supporting the concur-rent validity of the VCR1 and NCR1.The first data set was collected trialversions of these two tests whichcontained each of the stempassages/tables which appear in thefinal tests along with approximately80% of the final items. This datawas originally collected as part of thetest construction process in order tocheck that the trial items we hadconstructed were measuring reason-ing ability, and not some otherconstruct (e.g. reading ability,numerical checking etc.). This isparticularly important whenconstructing critical reasoning testsas it is easy to construct items whichare better measures of checkingability than they are of reasoningability. That is to say, items whichsimply require carefully scanningand memorising the text in order tosuccessfully complete them, ratherthan having to correctly drawslogical inferences from the text. Ascan be seen from the table, our trialitems clearly appear to be measuringability, as scores on both the VCR1and NCR1 are strongly correlatedwith the AH5.
Table 21 presents the correlationsbetween the Verbal/Numerical sub-scale of the AH5 and the finalversion of the two critical reasoningtests. This data therefore providesevidence in support of the fact thatthe final versions of these two tests
dn
measure reasoning ability rather thansome other construct (i.e. verbal ornumerical checking ability). As wasnoted above, when developing criticalreasoning tests it is particularly impor-tant to demonstrate that the tests aremeasuring reasoning ability, and notchecking ability.
The above data clearly demon-strates, as does the previous data set,that both the VCR1 & NCR1 aremeasuring reasoning ability. The sizeof these correlations indicate that theVerbal/Numerical sub-scale of theAH5 and the two critical reasoningtests share no more than 35% ofcommon variance, clearly demonstrat-ing that these tests are measuringdifferent, but related, constructs. Thisis what we would predict, given thatthe VCR1 & NCR1 were developed tomeasure critical reasoning, rather thanbe ‘pure’ measures of mental ability, orintelligence. Given the nature of criti-cal reasoning, we would expect acandidate’s scores on these tests notonly to reflect general reasoning abilityor intelligence (g), but also to reflectverbal and numerical comprehension,reading ability, reading speed andnumerical ability and precision.
Technical Test Battery
MRT2
SRT2
VAC2
VR2
.45
.35
.34
NR2
.45
.47
.40
AR2
.38
.46
.40
Table 18: Pearson Correlations of GRT2 withTechnical test Battery
CTB2 Sub-scales
Office Arithmetic (NA2)
Clerical Checking (CC2)
Spelling (SP2)
VR2
.51
.37
.34
Table 19: Pearson Correlations ofGRT2 with Clerical Test Battery
CRTB
VRC1
NRC1
AH5
.51
.52
Table 20: Product-moment Correlationsbetween the Experimental Versions of theVCR1 & NCR1 and the Verbal NumericalSub-scale of the AH5
CRTB
VRC1
NRC1
AH5
.60
.51
Table 21: Product-moment Correlationsbetween VCR1 & NCR1 and AH5Verbal/Numerical
do
RELATIONSHIP BETWEEN THECRTB AND WATSON-GLASER CRITICAL THINKING APPRAISAL
Table 22 details the relationshipbetween the CRTB and the Watson-Glaser Critical Thinking Appraisal.The relationship between the totalscore on CRTB and W-GCTA ismoderate, although this may be dueto its absence of numerical content.However, unexpectedly, the CRTBVerbal sub-scale does not appear tohave a higher correlation with the W-GCTA than the numerical. Insummary, while CRTB does notappear to be measuring exactly thesame construct as the W-GCTA, thedomains do overlap to such anextent that this provides someevidence that the CRTB is a measureof critical thinking.
THE RELATIONSHIP BETWEENTHE CRTB AND THE MULTI-DIMENSIONAL APTITUDE BATTERY
The CRTB was correlated withJackson’s MAB to assess the relativepositioning of the CRTB within abroad range of abilities. Table 23details the relationship between eachMAB sub-scale and the Verbal andNumerical components of the CRTBas well as the CRTB total score. TheCRTB total correlates .60 with theMAB total and is equally related toboth the MAB Verbal andPerformance scales (.57 and .52).More specifically at the sub-scalelevel, the CRTB total, relates signifi-cantly to three of each of the Verbaland Performance sub-scales. It ismore strongly related to theInformation, Arithmetic and ObjectPerception sub-scales (.60, .58, .58).This it would appear that the CRTBtotal measures a composite of bothVerbal and Performance abilities.
When the CRTB is divided into itssub-scales, the Verbal does notappear to be strongly related to theMAB Total but is related to the MABverbal scales in particular,Information and Vocabulary.
As expected, the Numericalsubscale relates more closely with theMAB Performance scale (.48) andalso correlates significantly withMAB Arithmetic and Spatial as itdoes with the Verbal which can beexpected by the higher verbalcontent in VCR1 than more classicmeasures of verbal ability.
dp
Verbal
Numerical
CRTB Total
WGCTA
.38
.38
.57
Table 22: Product-moment Correlationsbetween VCR1 & NCR1 and WGCTA
MAB Total
MAB Performance
MAB Verbal
Information
Comprehension
Arithmetic
Similarities
Vocabulary
Digit Symbol
Picture Completion
Spatial
Picture Arrangement
Object Assembly
Verbal
.24
.04
.43
.29
.25
.24
.22
.32
.09
.14
.19
.15
.09
Numerical
.48
.48
.44
.32
.44
.45
.33
.27
.37
.38
.50
.34
.23
Total
.60
.52
.57
.60
.52
.58
.39
.40
.39
.44
.58
.42
.44
Table 23: Product-moment Correlations between VCR1 &NCR1 and MAB (n=154)
dqCRITERION-RELATEDVALIDITY
In this section, we provide details ofnumber of studies in which thereasoning tests have been used aspart of a pilot study on a sample ofjob incumbents on whom perfor-mance data was available.
INSURANCE SALES
A sample of 86 Telephone SalesPersonnel with a leading MotorInsurance group completed theGRT2 as part of a validation study.The results of the GRT2 were corre-lated with a number of performancecriteria as detailed in Table 24.
The pattern of correlationssuggests that while there is a rela-tionship between GRT2 and perfor-mance measures, this is not alwaysin the anticipated direction. In factsome notable negative correlationswere observed, indicating that forsome performance criteria, higherreasoning ability may be less desir-able. However, the strongest correla-tions were found between GRT2 sub-scales and ‘Ins_Nce’ and the VR2with overall sales. The consistentnegative correlations with ‘Sales PH’would require further examination.
BANKING
A sample of 118 retail bankerscompleted the GRT2 and a personal-ity measure OPP as part of a concur-rent validation exercise. Participantswere rated on a range of competen-cies which focused primarily onpersonal qualities as opposed to abil-ities. As expected, GRT2 failed torelate strongly to the overall compe-tency rating which based on acomposite of ratings covering suchdiverse areas as Orderliness,Planning, Organising, Teamwork etc.
GRT2 did correlate with thoseratings which were associated withskill areas namely, numerical andsoftware related work (see Table 25).
SERVICE ENGINEERS
A leading International Crane &heavy lifting equipment servicingcompany tested a sample of 46service engineers on the GRT2battery and OPP. Their overallperformance was rated by supervi-sors on a behaviourally anchored fivepoint scale. From Table 26, theVerbal (VR2) is found to be stronglyrelated to rated performance (r=.46)and the Abstract (AR2) relatingmoderately with the same.
PRINTERS
A major local newspaper group withthe largest number of local titles inthe United Kingdom sought toexamine whether tests could predictthe job performance of experiencedprinters. A sample of 70 completedthe GRT2 battery as well as anumber of other measures includingthe OPP (Occupational PersonalityProfile) and TTB (Technical TestBattery). Each of the group wereassessed on a number of perfor-mance criteria by supervisors. Inaddition, test data were correlatedwith the results of a job sample printtest which was administered at selec-tion stage. Table 27 details theresults of this study.
Some noteworthy correlationswere registered with GRT2 andperformance measures. Firstly,overall performance is highly corre-lated with the Abstract (AR2) butalso moderately with the Verbal andNumerical sub-scales. The job
dr
sample criterion measure generallyregisters higher correlations with eachof the GRT2 sub-scales, reaching .41with the Abstract (AR2). Perhaps moresurprising, the Abstract correlates .56with a supervisor’s rating of Initiative,consistent with the behaviouraldescriptions used for this performancerating, which point to not simplytaking initiative but being able tosuccessfully resolve problem situations.
The single totally objectivelyderived performance measure wastime-keeping, which was based on anelectronic time and attendance record-ing system. Only one significant corre-lation was observed with Abstract(AR2) although this was negative,suggesting that those with higherability tended to have a poorer timeand attendance record.
Sales PH
Conversions%
Ins_Nce
Sales
Verbal
-.26
-.20
.32
.27
Numerical
-.27
-.21
.25
.14
Abstract
-.27
-.29
.33
.13
Table 24: Pearson Product Moment Correlations withGRT2 and job performance in an Insurance Setting
Sales PH Sales per hourConversions % Percent of Conversions from other policiesIns_NCE Composite Training Outcome MeasureSales Total Value of Policies sold
GRT2
VR2
NR2
AR2
PERF_ARI
.14
.29
.31
PERF_SOF
.03
.32
.28
Competency
-.02
.12
.01
Table 25: Pearson Product Moment Correlations withGRT2 and Performance criteria in Banking (N=118)
GRT2
Measure
Verbal
Numerical
Abstract
Overall
Performance
.46
-.24
.28
Table 26: Pearson ProductMoment correlations betweenGRT2 & Service EngineerPerformance
Criterion
Overall Performance
Performance Job Sample
Initiative
Time Keeping
Verbal
.26
.33
.40
_
Numerical
.28
.30
.44
_
Abstract
.36
.41
.56
.32
Table 27: Correlations Between GRT2 & Printer PerformanceCriteria (N=70)
ds
FINANCIAL SALESCONSULTANTS
A sample of 100 trainee FinancialConsultants from a major financialservices group completed the GRT2as part of a validation study. Table28 details the correlations betweenGRT2 and end of year examinations.
The results indicate that theNumerical (NR2) appears to be thebest predictor of examination perfor-mance with correlations (up to .46),although the Abstract (AR2) alsoregisters some highly notable corre-lations (up to .44). Only the Verbal(VR2) fails to relate to any of theexamination results which is perhapssomewhat unexpected.
TRAINING APPLICANTS FORCAR COMPONENT TRAINING COURSE
A large training company used theGeneral Reasoning Test to investi-gate the ability profiles of success-ful/non-successful applicants fortraining on a car components assem-bly task. The criterion of successused was only in part determined bytraining outcomes, with other factorsalso played a part, such as perceivedattitude, temperament etc. A sampleof 150 applicants was used for thestudy.
Both the Verbal (VR2) andAbstract (AR2) register moderatecorrelations with success on theprogramme. The numerical fails torelate with this criterion (see Table29).
GRT2 Sub-scale
Verbal
Numerical
Abstract
Criterion
Protection Clusters Result
Pension exam Results
Seller Induction Exam Results
Aggregate Result
Financial Planning Certificate Result
VR2
.10
.04.
.18
.11
.13
NR2
.31
.40
.26
.46
.44
AR2
.35
.32
.32
.44
.42
Table 28: Correlations between GRT2 & Proficiency Criteria
Success
.27
.16
.30
Table 29: Correlations betweenOPP & Successful Applicant forComponent Course
5ADMINISTRATIONINSTRUCTIONS● CRTB Administration Instructions
● GRT1 Administration Instructions
● GRT2 Administration Instructions
ekBEFORE STARTING THE QUESTIONNAIRE
Put candidates at their ease by giving information about yourself, thepurpose of the questionnaire, the timetable for the day, if this is part of awider assessment programme, and how the results will be used and who willhave access to them. Ensure that you and other administrators have switchedoff mobile phones etc.
The instructions below should be read out verbatim and the same scriptshould be followed each time the CRTB is administered to one or morecandidates. Instructions for the administrator are printed in ordinary type.Instructions designed to be read aloud to candidates are a larger typefacewith speech marks.
If this is the first or only questionnaire being administered give an introduc-tion as per or similar to the following example (prepare an amendment if notadministering all tests in this battery):
‘From now on, please do not talk among yourselves, butask me if anything is not clear. Please ensure that anymobile telephones, pagers or other potential distractions areswitched off completely. We shall be doing two tests: theVerbal Critical Reasoning Test, which takes 20 minutes andthe Numerical Critical Reasoning Test, which takes 30minutes. During the test I shall be checking to make surethat you are not making any accidental mistakes whenfilling in the answer sheet. I will not be checking yourresponses to see if you are answering correctly or not.’
WARNING: It is most important that answer sheets do not go astray. Theyshould be counted out at the beginning of the test and counted in again atthe end.
Continue by using the instructions EXACTLY as given. Say:
DISTRIBUTE THE ANSWER SHEETS
Then ask:
‘Has everyone got two sharp pencils, an eraser, some roughpaper and an answer sheet. Please note that the answerboxes are in columns (indicate) and remember, do not writeon the booklets.’
Rectify any omissions, then say:
CRTB ADMINISTRATIONINSTRUCTIONS
el‘Print your surname, first name and title clearly on the lineprovided, followed by your age and sex. Please inserttoday’s date which is [ ] on the “Comments” line’
Walk around the room to check that the instructions are being followed.
WARNING: It is vitally important that test booklets do not go astray. Theyshould be counted out at the beginning of the session and counted in again atthe end.
DISTRIBUTE THE BOOKLETS WITH THE INSTRUCTION:
‘Please do not open the booklet until instructed.’
Remembering to read slowly and clearly, go to the front of the group and say:
‘Please open the booklet at Page 2 and follow the instruc-tions for this test as I read them aloud.’
Pause to allow booklets to be opened.
‘In this test you have to draw inferences from shortpassages of text. You will be presented with a passage(there are 6 pieces of text in all) followed by a number ofstatements. Your task is to decide whether or not eachstatement can be inferred from the passage.
For each statement you will be asked to decide which of thefive response categories on Page 3 correctly describeswhether the statement can be inferred from the passage.
Mark your answer by filling in the appropriate box on youranswer sheet that corresponds to your choice. On Page 3 ofthis question booklet there are some example questions.Once you have fully read the instructions you will have achance to complete the example questions in order to makesure that you understand the test.
Please read the instructions and categories.’
Pause while the candidates read the instructions, then say:
‘Please attempt the example questions now.’
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatno-one is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two and a half minutes) give theanswers as follows:
em‘The correct response to Example 1 is Probably False.It is not explicitly stated within the text that voters areconcerned about the economy.
The correct response to Example 2 is Definitely TrueIt is explicitly stated that economic policies are often deter-mined by short term electoral demands.
The correct response to Example 3 is Definitely FalseIt explicitly states within the text that election results canbe predicted from economic indicators.’
Check for understanding, then say:
‘Time is short so when you begin the timed test work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice, and fill in your new choice of answer.
There are 6 passages of text and a total of 36 questions.You have 20 minutes in which to answer the questions.
If you reach the “End of Test” before time is called youmay review your answers if you wish.
If you have any questions please ask now, as I will not beable to answer questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 20 minutes, say:
‘Please now turn to Page 10 which is a blank page.’
NB: If this is the final test to be used in this battery, instead of the aboveline, please turn to the instructions in the Ending Test Session box on page45 of this document
enYou should intervene if candidates continue after this point.
Then say:
‘We are now ready to start the next test. Has everybody stillgot two sharpened pencils, an eraser and some unusedrough paper?’
If not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Check for understanding, then, remembering to read slowly and clearly, go tothe front of the group and say:
‘Please turn to Page 12 of the booklet and follow theinstructions for this test as I read them aloud.’ (Pause toallow page to be found).
In this test you have to draw inferences from numericalinformation which is presented in tabular form.
You will be presented with a numerical table and asked anumber of questions about this information. You will thenhave to select the correct answer to each question from oneof six possible choices. One and only one answer is correctin each case.
Mark your answer by filling in the appropriate box on youranswer sheet that corresponds to your choice.
You now have a chance to complete the example questionson Pages 13 in order to make sure that you understand thetest.
Please attempt the example questions now, using the roughpaper if necessary.’
eoWhile the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatno-one is looking at the actual test items during the example session. Whenall have finished (allow a maximum of three minutes) give the answers asfollows:
‘Please follow the answers and explanations carefully.
The correct answer to Example 1 is Design (answer no. 5).Amongst women, design was consistently chosen by thelowest percentage as the most important feature of a car.
The correct answer to Example 2 is Economy (answer no.2).Of all the features of a car, economy is the only feature forwhich the ratings increase according to age category.
The correct answer to Example 3 is 10.4 (answer no.5).Of males below 30, 5% identified safety and 52% identifiedperformance as the most important feature of a car. 5 over52 is 10.4.
Please do not turn over the page yet’
Then say:
‘Time is short so when you begin the timed test work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice, and fill in your new choice of answer.
There are 5 tables of information and a total of 25 ques-tions. You have 30 minutes in which to answer the ques-tions.
If you reach the “End of Test” before time is called youmay review your answers if you wish, but do not go back tothe Verbal Reasoning Test.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
ep
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 30 minutes:
ENDING THE TEST SESSION
Say:
‘Stop now please and close your booklet’
You should intervene if candidates continue after this point.
COLLECT THE ANSWER SHEETS AND THE TEST BOOKLETS,ENSURING THAT ALL MATERIALS ARE RETURNED (COUNTBOOKLETS AND ANSWER SHEETS)
Then say:
‘Thank you for completing the Critical Reasoning TestBattery.’
eqBEFORE STARTING THE QUESTIONNAIRE
Put candidates at their ease by giving information about yourself, thepurpose of the questionnaire, the timetable for the day, if this is part of awider assessment programme, and how the results will be used and who willhave access to them. Ensure that you and other administrators have switchedoff mobile phones etc.
The instructions below should be read out verbatim and the same scriptshould be followed each time the GRT1 is administered to one or morecandidates. Instructions for the administrator are printed in ordinary type.Instructions designed to be read aloud to candidates are a larger typefacewith speech marks.
If this is the first or only questionnaire being administered, give an introduc-tion as per or similar to the following example (prepare an amendment if notadministering all tests in this battery):
‘From now on, please do not talk among yourselves, butask me if anything is not clear. Please ensure that anymobile telephones, pagers or other potential distractions areswitched off completely. We shall be doing three tests:verbal, numerical and abstract reasoning. The tests take 8,10 and 10 minutes respectively to complete. During the testI shall be checking to make sure that you are not makingany accidental mistakes when filling in the answer sheet. Iwill not be checking your responses to see if you areanswering correctly or not.’
WARNING: It is most important that answer sheets do not go astray. Theyshould be counted out at the beginning of the test and counted in again atthe end.
Continue by using the instructions EXACTLY as given. Say:
DISTRIBUTE THE ANSWER SHEETS
Then ask:
‘Has everyone got two sharp pencils, an eraser, some roughpaper and an answer sheet.’
Rectify any omissions, then say:
‘Print your surname, first name and title clearly on the lineprovided and indicate your title, sex and age by ticking theappropriate boxes. Please insert today’s date which is [ ]’
GRT1 ADMINISTRATIONINSTRUCTIONS
erWalk around the room to check that the instructions are being followed.
WARNING: It is vitally important that test booklets do not go astray. Theyshould be counted out at the beginning of the session and counted in again atthe end.
DISTRIBUTE THE BOOKLETS WITH THE INSTRUCTION:
‘Please do not open the booklet until instructed.’
Remembering to read slowly and clearly, go to the front of the group and say:
‘Please open the booklet at Page 2 and follow the instruc-tions for this test as I read them aloud.’
Pause to allow booklets to be opened.
‘This test is designed to assess your understanding of wordsand relationships between words. Each question has sixpossible answers. One and only one is correct in each case.Mark your answer by filling in the number box on youranswer sheet that corresponds to your choice. You nowhave a chance to complete the four example questions onPage 3 in order to make sure that you understand the test.
Please attempt the example questions now, marking youranswers in boxes E1 to E4.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 2, sick means thesame as ill.
The answer to Example 2 is number 3, you drive a car andfly an aeroplane.
The answer to Example 3 is number 5, wood is the odd oneout.
The answer to Example 4 is number 4, as both heavy andlight have a relationship to weight.
Is everyone clear about the examples?’
esThen say:
‘REMEMBER:
Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, simply erase your firstchoice and fill in your new answer.
There are a total of 30 questions and you have 8 minutes inwhich to answer them.
If you reach the end before time is called you may reviewyour answers if you wish.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 8 minutes, say:
‘Stop now please and turn to Page 12.’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 52 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.
Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
etIf not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 12 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘This test is designed to assess your ability to work withnumbers. Each question has six possible answers. One andonly one is correct in each case. Mark your answer byfilling in the appropriate box that corresponds to yourchoice on the answer sheet.’
You now have a chance to complete the four example questions on Page 13in order to make sure that you understand the test. Please attempt theexample questions now, marking your answers in the example boxes.
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, the sequence goesup in twos.
The answer to Example 2 is number 4, as all other frac-tions can be reduced further.
The answer to Example 3 is number 2, 100 is 10 times 10.
The answer to Example 4 is number 5, the journey willtake 1 hour and 30 minutes.
Is everyone clear about the examples?’
fkThen say:
‘Time is short, so when you begin the timed test work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called you may reviewyour answers to the numerical test if you wish, but do notgo back to the verbal test.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions, appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes, say:
‘Stop now please and turn to Page 20’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 52 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
flIf not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 20 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘In this test you will have to work out the relationshipbetween abstract shapes and patterns.
Each question has six possible answers. One and only one iscorrect in each case. Mark your answer by filling in theappropriate box that corresponds to your chosen answer onyour answer sheet. You now have a chance to complete thethree example questions on Page 21 in order to make surethat you understand the test.
Please attempt the example questions now, marking youranswers in the example boxes.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished, (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, as the series alter-nates between 2 and 4 squares as does the direction of thetwo squares which return to their original position.
The answer to Example 2 is number 4, as all of the otheroptions have an open side to one of the boxes.
The answer to Example 3 is number 6, as this is a mirrorimage of the pattern.
Is everyone clear about the examples?’
fmThen say:
‘Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice, and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called, you may reviewyour answers to the abstract test, but do not go back to theprevious tests.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes:
ENDING THE TEST SESSION
Say:
‘Stop now please and close your booklet’
You should intervene if candidates continue after this point.
COLLECT THE ANSWER SHEETS AND THE TEST BOOKLETS,ENSURING THAT ALL MATERIALS ARE RETURNED (COUNTBOOKLETS AND ANSWER SHEETS)
Then say:
‘Thank you for completing the Graduate Reasoning Test.’
fnBEFORE STARTING THE QUESTIONNAIRE
Put candidates at their ease by giving information about yourself, thepurpose of the questionnaire, the timetable for the day, if this is part of awider assessment programme, and how the results will be used and who willhave access to them. Ensure that you and other administrators have switchedoff mobile phones etc.
The instructions below should be read out verbatim and the same scriptshould be followed each time the GRT2 is administered to one or morecandidates. Instructions for the administrator are printed in ordinary type.Instructions designed to be read aloud to candidates are a larger typefacewith speech marks.
If this is the first or only questionnaire being administered, give an introduc-tion as per or similar to the following example (prepare an amendment if notadministering all tests in this battery):
‘From now on, please do not talk among yourselves, butask me if anything is not clear. Please ensure that anymobile telephones, pagers or other potential distractions areswitched off completely. We shall be doing three tests:verbal, numerical and abstract reasoning. The tests take 8,10 and 10 minutes respectively to complete. During the testI shall be checking to make sure that you are not makingany accidental mistakes when filling in the answer sheet. Iwill not be checking your responses to see if you areanswering correctly or not.’
WARNING: It is most important that answer sheets do not go astray. Theyshould be counted out at the beginning of the test and counted in again atthe end.
Continue by using the instructions EXACTLY as given. Say:
DISTRIBUTE THE ANSWER SHEETS
Then ask:
‘Has everyone got two sharp pencils, an eraser, some roughpaper and an answer sheet.’
Rectify any omissions, then say:
‘Print your surname, first name and title clearly on the lineprovided and indicate your title, sex and age by ticking theappropriate boxes. Please insert today’s date which is [ ].’
GRT2 ADMINISTRATIONINSTRUCTIONS
foWalk around the room to check that the instructions are being followed.
WARNING: It is vitally important that test booklets do not go astray. Theyshould be counted out at the beginning of the session and counted in again atthe end.
DISTRIBUTE THE BOOKLETS WITH THE INSTRUCTION:
‘Please do not open the booklet until instructed.’
Remembering to read slowly and clearly, go to the front of the group and say:
‘Please open the booklet at Page 2 and follow the instruc-tions for this test as I read them aloud.’
Pause to allow booklets to be opened.
‘This test is designed to assess your understanding of wordsand relationships between words. Each question has sixpossible answers. One and only one is correct in each case.Mark your answer by filling in the number box on youranswer sheet that corresponds to your choice. You nowhave a chance to complete the four example questions onPage 3 in order to make sure that you understand the test.
Please attempt the example questions now, marking youranswers in boxes E1 to E4.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 2, sick means thesame as ill.
The answer to Example 2 is number 3, you drive a car andfly an aeroplane.
The answer to Example 3 is number 5, wood is the odd oneout.
The answer to Example 4 is number 5, as dark means theopposite of light.
Is everyone clear about the examples?’
fpThen say:
‘REMEMBER:
Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, simply erase your firstchoice and fill in your new answer.
There are a total of 35 questions and you have 8 minutes inwhich to answer them.
If you reach the end before time is called you may reviewyour answers if you wish.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 8 minutes, say:
‘Stop now please and turn to Page 12.’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 59 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.
Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
fqIf not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 12 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘This test is designed to assess your ability to understandnumbers and the relationship between numbers. Eachquestion has six possible answers. One and only one iscorrect in each case. Mark your answer by filling in theappropriate box that corresponds to your chosen answer onthe answer sheet.
You now have a chance to complete the four example questions on Page 13in order to make sure that you understand the test. Please attempt theexample questions now, marking your answers in the example boxes.
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, the sequence goesup in twos.
The answer to Example 2 is number 4, as all other frac-tions can be reduced further.
The answer to Example 3 is number 2, 100 is 10 times 10.
The answer to Example 4 is 5, the journey will take 1 hour30 minutes.
Is everyone clear about the examples?’
frThen say:
‘Time is short, so when you begin the timed test work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called you may reviewyour answers to the numerical test if you wish, but do notgo back to the verbal test.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions, appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes, say:
‘Stop now please and turn to Page 20’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 59 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.
Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
fsIf not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 20 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘In this test you will have to work out the relationshipbetween shapes and figures.
Each question has six possible answers. One and only one iscorrect in each case. Mark your answer by filling in theappropriate box that corresponds to your chosen answer onyour answer sheet. You now have a chance to complete thethree example questions on Page 21 in order to make surethat you understand the test.
Please attempt the example questions now, marking youranswers in the example boxes.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished, (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, as the series alter-nates between 2 and 4 squares as does the direction of thetwo squares which return to their original position.
The answer to Example 2 is number 4, as all of the otheroptions have an open side to one of the boxes.
The answer to Example 3 is number 6, as this is a mirrorimage of the pattern.
Is everyone clear about the examples?’
ftThen say:
‘Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice, and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called, you may reviewyour answers to the abstract test, but do not go back to theprevious tests.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes:
ENDING THE TEST SESSION
Say:
‘Stop now please and close your booklet’
You should intervene if candidates continue after this point.
COLLECT THE ANSWER SHEETS AND THE TEST BOOKLETS,ENSURING THAT ALL MATERIALS ARE RETURNED (COUNTBOOKLETS AND ANSWER SHEETS)
Then say:
‘Thank you for completing the General Reasoning Test.’
gk
6REFERENCESBinet. A (1910) Les idées modernessur les enfants Paris: E. Flammarion.
Cronbach L.J. (1960) Essentials ofPsychological Testing (2nd Edition)New York: Harper.
Galton F. (1869) Heriditary GenuisLondon: MacMillan.
Gould, S.J. (1981). The Mismeasureof Man. Harmondsworth, Middlesex:Pelican.
Heim, A.H. (1970). Intelligence andPersonality. Harmondsworth,Middlesex: Penguin.
Heim, A.H., Watt, K.P. andSimmonds, V. (1974). AH2/AH3Group Tests of General Reasoning;Manual. Windsor: NFER Nelson.
Jackson D.N. (1987) User’s Manualfor the Multidimensional AptitudeBattery London, Ontario: ResearchPsychologists Press.
Johnson, C., Blinkhorn, S., Wood, R.and Hall, J. (1989). ModernOccupational Skills Tests: User’sGuide. Windsor: NFER-Nelson.
Budd R.J. (1991) Manual for theClerical Test Battery: Letchworth,Herts UK: Psytech InternationalLimited
Budd R.J. (1993) Manual for theTechnical Test Battery: Letchworth,Herts UK: Psytech InternationalLimited
Stern W (1912) PsychologischeMethoden der Intelligenz-Prüfung.Leipzig, Germany: Barth
Terman, L.M. et. al., (1917). TheStanford Revision of the Binet-Simonscale for measuring intelligence.Baltimore: Warwick and York.
Watson & Glaser (1980) Manual forthe Watson-Glaser Critical ThinkingAppraisal Harcourt BraceJovanovich: New York
Yerkes, R.M. (1921). Psychologicalexamining in the United States army.Memoirs of the National Academy ofSciences, 15.
INTERNATIONAL
general,critical &graduateTEST BATTERYadministration instructions
2instructions
BEFORE STARTING THE QUESTIONNAIRE
Put candidates at their ease by giving information about yourself, thepurpose of the questionnaire, the timetable for the day, if this is part of awider assessment programme, and how the results will be used and who willhave access to them. Ensure that you and other administrators have switchedoff mobile phones etc.
The instructions below should be read out verbatim and the same scriptshould be followed each time the GRT1 is administered to one or morecandidates. Instructions for the administrator are printed in ordinary type.Instructions designed to be read aloud to candidates are a larger typefacewith speech marks.
If this is the first or only questionnaire being administered, give an introduc-tion as per or similar to the following example (prepare an amendment if notadministering all tests in this battery):
‘From now on, please do not talk among yourselves, butask me if anything is not clear. Please ensure that anymobile telephones, pagers or other potential distractions areswitched off completely. We shall be doing three tests:verbal, numerical and abstract reasoning. The tests take 8,10 and 10 minutes respectively to complete. During the testI shall be checking to make sure that you are not makingany accidental mistakes when filling in the answer sheet. Iwill not be checking your responses to see if you areanswering correctly or not.’
WARNING: It is most important that answer sheets do not go astray. Theyshould be counted out at the beginning of the test and counted in again atthe end.
Continue by using the instructions EXACTLY as given. Say:
DISTRIBUTE THE ANSWER SHEETS
Then ask:
‘Has everyone got two sharp pencils, an eraser, some roughpaper and an answer sheet.’
Rectify any omissions, then say:
‘Print your surname, first name and title clearly on the lineprovided and indicate your title, sex and age by ticking theappropriate boxes. Please insert today’s date which is [ ]’
GRT1 ADMINISTRATIONINSTRUCTIONS
3instructions
Walk around the room to check that the instructions are being followed.
WARNING: It is vitally important that test booklets do not go astray. Theyshould be counted out at the beginning of the session and counted in again atthe end.
DISTRIBUTE THE BOOKLETS WITH THE INSTRUCTION:
‘Please do not open the booklet until instructed.’
Remembering to read slowly and clearly, go to the front of the group and say:
‘Please open the booklet at Page 2 and follow the instruc-tions for this test as I read them aloud.’
Pause to allow booklets to be opened.
‘This test is designed to assess your understanding of wordsand relationships between words. Each question has sixpossible answers. One and only one is correct in each case.Mark your answer by filling in the number box on youranswer sheet that corresponds to your choice. You nowhave a chance to complete the four example questions onPage 3 in order to make sure that you understand the test.
Please attempt the example questions now, marking youranswers in boxes E1 to E4.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 2, sick means thesame as ill.
The answer to Example 2 is number 3, you drive a car andfly an aeroplane.
The answer to Example 3 is number 5, wood is the odd oneout.
The answer to Example 4 is number 4, as both heavy andlight have a relationship to weight.
Is everyone clear about the examples?’
4instructions
Then say:
‘REMEMBER:
Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, simply erase your firstchoice and fill in your new answer.
There are a total of 30 questions and you have 8 minutes inwhich to answer them.
If you reach the end before time is called you may reviewyour answers if you wish.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 8 minutes, say:
‘Stop now please and turn to Page 12.’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 14 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
5instructions
If not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 12 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘This test is designed to assess your ability to work withnumbers. Each question has six possible answers. One andonly one is correct in each case. Mark your answer byfilling in the appropriate box that corresponds to yourchoice on the answer sheet.’
You now have a chance to complete the four example questions on Page 13in order to make sure that you understand the test. Please attempt theexample questions now, marking your answers in the example boxes.
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, the sequence goesup in twos.
The answer to Example 2 is number 4, as all other frac-tions can be reduced further.
The answer to Example 3 is number 2, 100 is 10 times 10.
The answer to Example 4 is number 5, the journey willtake 1 hour and 30 minutes.
Is everyone clear about the examples?’
6instructions
Then say:
‘Time is short, so when you begin the timed test work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called you may reviewyour answers to the numerical test if you wish, but do notgo back to the verbal test.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions, appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes, say:
‘Stop now please and turn to Page 20’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on the finalpage of this document. If you are skipping a test, please find the appropriatetest bookmark for your next test in the margin of the page and replace theabove line as necessary.
You should intervene if candidates continue after this point.Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
7instructions
If not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 20 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘In this test you will have to work out the relationshipbetween abstract shapes and patterns.
Each question has six possible answers. One and only one iscorrect in each case. Mark your answer by filling in theappropriate box that corresponds to your chosen answer onyour answer sheet. You now have a chance to complete thethree example questions on Page 21 in order to make surethat you understand the test.
Please attempt the example questions now, marking youranswers in the example boxes.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished, (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, as the series alter-nates between 2 and 4 squares as does the direction of thetwo squares which return to their original position.
The answer to Example 2 is number 4, as all of the otheroptions have an open side to one of the boxes.
The answer to Example 3 is number 6, as this is a mirrorimage of the pattern.
Is everyone clear about the examples?’
8instructions
Then say:
‘Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice, and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called, you may reviewyour answers to the abstract test, but do not go back to theprevious tests.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes:
ENDING THE TEST SESSION
Say:
‘Stop now please and close your booklet’
You should intervene if candidates continue after this point.
COLLECT THE ANSWER SHEETS AND THE TEST BOOKLETS,ENSURING THAT ALL MATERIALS ARE RETURNED (COUNTBOOKLETS AND ANSWER SHEETS)
Then say:
‘Thank you for completing the Graduate Reasoning Test.’
15instructions
BEFORE STARTING THE QUESTIONNAIRE
Put candidates at their ease by giving information about yourself, thepurpose of the questionnaire, the timetable for the day, if this is part of awider assessment programme, and how the results will be used and who willhave access to them. Ensure that you and other administrators have switchedoff mobile phones etc.
The instructions below should be read out verbatim and the same scriptshould be followed each time the GRT2 is administered to one or morecandidates. Instructions for the administrator are printed in ordinary type.Instructions designed to be read aloud to candidates are a larger typefacewith speech marks.
If this is the first or only questionnaire being administered, give an introduc-tion as per or similar to the following example (prepare an amendment if notadministering all tests in this battery):
‘From now on, please do not talk among yourselves, butask me if anything is not clear. Please ensure that anymobile telephones, pagers or other potential distractions areswitched off completely. We shall be doing three tests:verbal, numerical and abstract reasoning. The tests take 8,10 and 10 minutes respectively to complete. During the testI shall be checking to make sure that you are not makingany accidental mistakes when filling in the answer sheet. Iwill not be checking your responses to see if you areanswering correctly or not.’
WARNING: It is most important that answer sheets do not go astray. Theyshould be counted out at the beginning of the test and counted in again atthe end.
Continue by using the instructions EXACTLY as given. Say:
DISTRIBUTE THE ANSWER SHEETS
Then ask:
‘Has everyone got two sharp pencils, an eraser, some roughpaper and an answer sheet.’
Rectify any omissions, then say:
‘Print your surname, first name and title clearly on the lineprovided and indicate your title, sex and age by ticking theappropriate boxes. Please insert today’s date which is [ ].’
GRT2 ADMINISTRATIONINSTRUCTIONS
16instructions
Walk around the room to check that the instructions are being followed.
WARNING: It is vitally important that test booklets do not go astray. Theyshould be counted out at the beginning of the session and counted in again atthe end.
DISTRIBUTE THE BOOKLETS WITH THE INSTRUCTION:
‘Please do not open the booklet until instructed.’
Remembering to read slowly and clearly, go to the front of the group and say:
‘Please open the booklet at Page 2 and follow the instruc-tions for this test as I read them aloud.’
Pause to allow booklets to be opened.
‘This test is designed to assess your understanding of wordsand relationships between words. Each question has sixpossible answers. One and only one is correct in each case.Mark your answer by filling in the number box on youranswer sheet that corresponds to your choice. You nowhave a chance to complete the four example questions onPage 3 in order to make sure that you understand the test.
Please attempt the example questions now, marking youranswers in boxes E1 to E4.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 2, sick means thesame as ill.
The answer to Example 2 is number 3, you drive a car andfly an aeroplane.
The answer to Example 3 is number 5, wood is the odd oneout.
The answer to Example 4 is number 5, as dark means theopposite of light.
Is everyone clear about the examples?’
17instructions
Then say:
‘REMEMBER:
Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, simply erase your firstchoice and fill in your new answer.
There are a total of 35 questions and you have 8 minutes inwhich to answer them.
If you reach the end before time is called you may reviewyour answers if you wish.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 8 minutes, say:
‘Stop now please and turn to Page 12.’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 21 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.
Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
18instructions
If not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 12 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘This test is designed to assess your ability to understandnumbers and the relationship between numbers. Eachquestion has six possible answers. One and only one iscorrect in each case. Mark your answer by filling in theappropriate box that corresponds to your chosen answer onthe answer sheet.
You now have a chance to complete the four example questions on Page 13in order to make sure that you understand the test. Please attempt theexample questions now, marking your answers in the example boxes.
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, the sequence goesup in twos.
The answer to Example 2 is number 4, as all other frac-tions can be reduced further.
The answer to Example 3 is number 2, 100 is 10 times 10.
The answer to Example 4 is 5, the journey will take 1 hour30 minutes.
Is everyone clear about the examples?’
Then say:
‘Time is short, so when you begin the timed test work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called you may reviewyour answers to the numerical test if you wish, but do notgo back to the verbal test.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions, appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes, say:
‘Stop now please and turn to Page 20’
NB: If this is the final test to be used in this battery, instead of the above line,please turn to the instructions in the Ending Test Session box on page 21 ofthis document. If you are skipping a test, please find the appropriate testbookmark for your next test in the margin of the page and replace the aboveline as necessary.
You should intervene if candidates continue after this point.
Then say:
‘We are now ready to start the next test. Has everyone stillgot two sharpened pencils, an eraser, some unused roughpaper?’
19instructions
20instructions
If not, rectify, then say:
‘The next test follows on the same answer sheet, pleaselocate the section now.’
Indicate section
Check for understanding, then remembering to read slowly and clearly, go tothe front of the group and say:
‘Please ensure that you are on Page 20 of the booklet andfollow the instructions for this test as I read them aloud.’
Pause to allow page to be found.
‘In this test you will have to work out the relationshipbetween shapes and figures.
Each question has six possible answers. One and only one iscorrect in each case. Mark your answer by filling in theappropriate box that corresponds to your chosen answer onyour answer sheet. You now have a chance to complete thethree example questions on Page 21 in order to make surethat you understand the test.
Please attempt the example questions now, marking youranswers in the example boxes.’
Indicate section
While the candidates are doing the examples, walk around the room to checkthat everyone is clear about how to fill in the answer sheet. Make sure thatnobody is looking at the actual test items during the example session. Whenall have finished, (allow a maximum of two minutes) give the answers asfollows:
‘The answer to Example 1 is number 5, as the series alter-nates between 2 and 4 squares as does the direction of thetwo squares which return to their original position.
The answer to Example 2 is number 4, as all of the otheroptions have an open side to one of the boxes.
The answer to Example 3 is number 6, as this is a mirrorimage of the pattern.
Is everyone clear about the examples?’
Then say:
‘Time is short, so when you begin the timed test, work asquickly and as accurately as you can.
If you want to change an answer, fully erase your firstchoice, and fill in your new choice of answer.
There are a total of 25 questions and you have 10 minutesin which to attempt them.
If you reach the end before time is called, you may reviewyour answers to the abstract test, but do not go back to theprevious tests.
If you have any questions please ask now, as you will not beable to ask questions once the test has started.’
Then say very clearly:
‘Is everyone clear about how to do this test?’
Deal with any questions appropriately, then, starting stop-watch or setting acount-down timer on the word BEGIN say:
‘Please turn over the page and begin.’
Answer only questions relating to procedure at this stage, but enter in theAdministrator’s Test Record any other problems which occur. Walk aroundthe room at appropriate intervals to check for potential problems.
At the end of the 10 minutes:
ENDING THE TEST SESSION
Say:
‘Stop now please and close your booklet’
You should intervene if candidates continue after this point.
COLLECT THE ANSWER SHEETS AND THE TEST BOOKLETS,ENSURING THAT ALL MATERIALS ARE RETURNED (COUNTBOOKLETS AND ANSWER SHEETS)
Then say:
‘Thank you for completing the General Reasoning Test.’
21instructions
top related