© cognimetrics, inc., 2005 prognost a web-based personnel assessment system

© CogniMetrics, Inc., 2005

PROGNOSTPROGNOST

A Web-Based Personnel Assessment System

Content Content navigatornavigator

PROGNOST: a web-based assessment system• What is PROGNOST?• For whom PROGNOST is designed

PROGNOST: Features• Quality Selection Decisions• Local Accessibility• User Languages: English and Spanish• Expandable with non web-based tests

PROGNOST: Applications• Selection of Terminal Control Room Operator• Selection of Executive Assistent• Selection of Retail & Specialty Product Marketing

PROGNOST: Specifications• IT Aptitude Battery• Wonderlic Personnel Test• Personality Characteristics Inventory• Decision Support System

Pricing

AvailabilityThe compass icon appears on many slides in the slide show. Click on one of its arrows to return to the navigator page.

PROGNOSTPROGNOST:: web-based assessment systemweb-based assessment system

PROGNOST a web-based assessment system for personnel selection. PROGNOST consists of:

low-cost customizable decision-support system.

standard (but expandable) battery of web-delivered tests.

What is PROGNOST?

PROGNOSTPROGNOST:: web-based assessment systemweb-based assessment system

PROGNOST is designed for companies that:

• give a high priority to the quality of their selection and promotion decisions.• expect accountability of human resources manage-ment.• have employees working in workplaces that are spread out over a region, nation, or internationally. • have employees with ethnic, cultural and linguistic different backgrounds.

RETURN

For whom PROGNOST is designed

The main goal of our web-based assessment system is to improve the quality of personnel decisions. We will introduce PROGNOST by discussing potential problems with the quality of personnel decisions.

PROGNOST PROGNOST featuresfeatures

Quality Selection Decisions

Psychological assessment is a systematic effort to collect and to combine relevant information for the purpose of diagnosis or prognosis of human behavior.


The quality of psychological assessment often is sub-optimal because of the following:

Most HR-professionals restrict the systematic effort part to collecting information (i.e., testing and interviewing). To combine the information experts use so-called “clinical judgment”.

Over 50 years of research has shown that combining this information using “clinical judgment” is highly inaccurate and subject to distortion.



Until recently no user-friendly system was available to replace “clinical judgment”.

Traditional statistical techniques were too restrictive and were not user-friendly.



In 2003 CogniMetrics, Inc. developed PROGNOST a method to build customized low-cost decision-support systems for selection decisions.

A decision support system is an expert system that uses all available scientific knowledge to predict success of current or future employees for a particular job or training, based on available information (such as their test scores).

What follows is a short introduction to PROGNOST:



collecting informationto predict probability of success

in job / training

combining this information into a single statement

expressing the probabilityof success

testsquestionnaires

decision-support system

RISK ASSESSMENTPER INDIVIDUAL

Psychological Assessment

testsquestionnaires

collecting informationto predict probability of success

in job / training

combining this information into a single statement

expressing the probabilityof success

decision-support system

RISK ASSESSMENTPER INDIVIDUAL

Psychological Assessment

s3s4

42.025.3

s3s4

46.020.6

s3s4

40.627.3

s3s4

43.318.0

s3s4

44.623.3

similarities

s1s2

4.6822.0

arithmetic

s1s2

3.3524.0

comprehension

s1s2

6.6720.7

inductive_reasoning

s1s2

1.3429.4

numbers

s1s2

4.6623.4

INT_factor

s1s2s3s4s5

4.7024.741.323.36.00

s3s4s5

42.025.36.00

s3s4s5

46.020.65.99

s3s4s5

40.627.34.65

s3s4s5

44.623.34.00

similarities

s1s2

4.6822.0

arithmetic

s1s2

3.3524.0

comprehension

s1s2

6.6720.7

numbers

s1s2

4.6623.4

PROGNOST PROGNOST decision support systemdecision support system

What do we need it for?

• Human experts (psychologists, medical doctors) are incapable of reliably combining information of multiple tests into a single decision outcome (e.g., a hire / no-hire).

• This problem is particularly serious when many information sources (such as tests) have to be considered.

s3s4s5

42.025.36.00

s3s4s5

46.020.65.99

s3s4s5

40.627.34.65

s3s4s5

44.623.34.00

similarities

s1s2

4.6822.0

arithmetic

s1s2

3.3524.0

comprehension

s1s2

6.6720.7

numbers

s1s2

4.6623.4

PROGNOSTPROGNOST decision support systemdecision support system

How does it work?

A decision-support system combines results of scientific studies, common-sense knowledge and observational evidence to come up with a prediction of the probability of success in training or job.

s3s4s5

42.025.36.00

s3s4s5

46.020.65.99

s3s4s5

40.627.34.65

s3s4s5

44.623.34.00

similarities

s1s2

4.6822.0

arithmetic

s1s2

3.3524.0

comprehension

s1s2

6.6720.7

numbers

s1s2

4.6623.4


Expert system(BN-model)

Theoreticalinsights

Empiricalobservations PREDICTIONS

Commonsense

How does it work?

s3s4s5

42.025.36.00

s3s4s5

46.020.65.99

s3s4s5

40.627.34.65

s3s4s5

44.623.34.00

similarities

s1s2

4.6822.0

arithmetic

s1s2

3.3524.0

comprehension

s1s2

6.6720.7

numbers

s1s2

4.6623.4


We build decision support systems using Bayesian Networks, an artificial intelligence technology.

Bayesian Networks propagate information derived from objective tests (and subjective ratings if required) to a selection or placement advise for an individual applicant.

Bayesian Networks are more powerful and more robust than traditional regression techniques.

How does it work?

s3s4s5

42.025.36.00

s3s4s5

46.020.65.99

s3s4s5

40.627.34.65

s3s4s5

44.623.34.00

similarities

s1s2

4.6822.0

arithmetic

s1s2

3.3524.0

comprehension

s1s2

6.6720.7

numbers

s1s2

4.6623.4


RETURN

PROGNOST produces the following output:

1. Odds of success: a simple quantitative index. If odds > 1.00 candidate is likely to succeed on the particular task, else (s)he is more likely to fail.

2. A more detailed report with the candidates’ strong and weak points.

How does it work?

TESTSTESTS collecting informationcollecting information

CogniMetrics, Inc. offers a standard battery of tests and questionnaires that together predict over 50 % of the variance in training and job performance.

• all tests are web delivered• languages: English, Spanish• economical: low assessment costs per candidate• the standard battery can be expanded or tests can be tailored to specific Alcoa needs.

The standard battery of PROGNOST administers two main groups of tests / questionnaires:

• Intelligence

• Personality traits

TESTSTESTS collecting informationcollecting information

TESTS TESTS intelligenceintelligence

Eighty years of selection research has proven tests of intelligence to be the strongest predictors of success in training and job performance. The standard battery comprises two different types of intelligence tests:

• Wonderlic Personnel Test (WPT - Wonderlic, Inc.)

• IT Aptitude Test (ITAB - CogniMetrics, Inc.)RETURN

IT Aptitude BatteryIT Aptitude Battery

General:

The ITAB tests measure an aspect of intelligence that has become increasingly important since the advent of Information Technology. ITAB tests measure the ability to interact efficiently with systems that have a dynamic of their own. Such systems vary in complexity from MS Office software tools to complex information systems monitoring industrial processes.

The cognitive skill required for using such devices is called procedural knowledge, or procedural skill.

IT Aptitude BatteryIT Aptitude Battery

Features:

The ITAB test reports deliver:• a multifaceted PERFORMANCE SCORE, measuring Procedural Knowledge Aptitude,

and scores on two causative factors:

• INTELLIGENCE: efficiency in information processing

• SYSTEMATIC APPROACH: steady approach and careful inferences.

RETURN

Wonderlic Personnel TestWonderlic Personnel Test

The Wonderlic Personnel Test (WPT) is a classical test of general cognitive ability (general intelligence).

It measures primarily so-called crystalyzed intelligence: a product of past learning.

Wonderlic Personnel TestWonderlic Personnel Test

Features:

• Short test of general intelligence (test time 12 minutes) with reliabilities and validities comparable with much longer tests.

• 50 items in ascending order of difficulty.

• Web delivered.

RETURN

TESTS TESTS personalitypersonality

Certain core personality traits are a predisposition to long-term commitments, such as needed to complete a training program..

PC InventoryPC Inventory

The Personality Characteristics Inventory evaluates candidates on five basic dimensions of personality:

• Conscientiousness• Extraversion• Agreeableness• Openness• Stability


The 150 items can be rearranged in several combinations so as to generate various scores:

• 12 subscales: efficiency, dependability, achievement striving, etc.

• success scales: commitment to work, integrity, learning, orientation, teamwork.

• occupational scales: manager, sales, production worker, driver, clerical employee.


Features:

• No time limit (generally takes 25-30 minutes to complete).

• 150 Questions.

• Valid, reliable, and legal.

RETURN

CogniMetrics’ tests are web-delivered, which makes test administration possible at virtually any place of the globe.

This is ideal for companies that have geographically spread work locations.

An additional advantage of our system is that it guarantees an equal weighing of test information throughout the company.

Local Accessibility


All tests of the proposed standard battery are available in both English and Spanish.

PROGNOST PROGNOST featuresfeatures PROGNOST PROGNOST featuresfeatures

User languages


Expandable system

PROGNOST can be designed to deal with information gathered through observation, interview, and paper-and-pencil tests.

For example, the HR specialist may want to add behavior ratings obtained from observing Assessment Center exercises.

PROGNOST can be made to accept this information and weigh it appropriately.

Applications Applications examplesexamples

PROGNOST can be designed to optimally predict any category of employment positions. CogniMetrics

• reviews empirical studies involving the prediction of success in those positions and estimates the population correlation between predictors and criterion.

• translates the outcomes into the structure of the Bayesian Network (conditional probabilities).

The following slides show a number current job openings in large San Antonio based companies (Alcoa, Inc.; H.E.B., VALERO Energy Corp.). The purpose is to demonstrate that PROGNOST with the ITAB tests has an edge in predicting success in work environments of the 21th century.


Terminal Control Room Operator(VALERO, Job ID: 5287-70000393)

Position description:Monitors and supports all load rack activities related to the terminal automation system. Monitors terminal automation systems and advise field personnel when a terminal is in need of attention. Tracks product movements to help reduce the chance of terminal outages or possible overfills. Supports troubleshooting of the terminal control system with the assistance of Information Services and/or field personnel. Etc.

Experience and Educational requirements: High school diploma or GED required. Some college background preferred. Minimum two years of experience in terminal operations or industry experience required. Basic knowledge of petroleum products and industry terminology required. Excellent PC skills, with proficiency in Microsoft products required. Effective analytical and problem-solving skills.

Relevant tests:The ITAB tests measure ability to deal with procedural knowledge, a crucial aptitude in work environments of the 21th century.


Selection of Executive Assistant(ALCOA, Job-ID: 2410BR)

Function description:• Must possess excellent computer skills (Microsoft Word, Excel, PowerPoint, Outlook etc.)• Responsible for travel planning of the Senior Management Team. • The chosen candidate must be able to multi-task, work with confidential information, be self motivated and a team player.• Excellent organizational skills required

Experience and Educational Requirements: high school degree or equivalent, bachelors preferred. Requires 10 years of general administrative and secretarial experience with emphasis in working independently towards results and goals.

Most relevant tests: ITAB tests measure a crucial aptitude for the 21th century work environment.


Retail & Specialty Product Marketing (VALERO, Job ID: 5359-22995)

Position Description:Performs maintenance and upkeep of Retail facilities. Diagnoses and repairs a variety of refrigeration equipment problems to include HVAC, walk-in coolers, icemakers, and freezers. Diagnoses and repairs a variety of merchandising and fast-food equipment problems such as coffee makers, … cappuccino machines, and related equipment. Maintains and repairs building structures, including roofing, plumbing, drywall, flooring, ceilings, glazing, doors, hardware, electrical, and related building items.

Experience and Educational requirements:High school diploma or GED required. Vocational training in craft-related areas required. Basic computer skills required. Must have experience in selecting proper tools and repair methods. Etc.

Relevant tests: Many jobs require technical troubleshooting (diagnoses and repair, see above), a form of procedural knowledge handling. See: ITAB tests.

PROGNOST PROGNOST SpecificationsSpecifications

PROGNOST consists of:

a Bayesian Network

a standard battery of tests: ITAB: IT Aptitude Battery WPT: Wonderlic Personnel Test PCI: Personality Characteristic Inventory

Measurement claim: procedural knowledge handling(aspect of general intelligence)

Number of tests: 2 (HTT and BT)Administration time: 10 minutes per test (average)Languages: English, Spanish, German,

DutchTest Publisher: CogniMetrics, Inc.Published: March 2004Price (if purchased separately): see: www.cognimetrics.com

Reliabilities: HTT (0.88 – 0.97)BT (0.80 – 0.87)

Validities:• construct validity studies: see: FOR MORE INFORMATION• predictive validity studies:currently under way with:

- Air Force Research Lab, Brooks City Base;

- Navy Directorate Selection and

Classification, Washington, D.C.

FOR MORE INFO...

• Download ITAB manuals: www.cognimetrics.com• Report on construct validity: www.internationalmta.org/2004 (paper 066; powerpoint presentation 066).

SpecificationsSpecifications Standard Test Battery Standard Test Battery

IT Aptitude Battery

Wonderlic Personnel Test

Measurement claim: general intelligence

Number of tests: 1 (50 questions)Administration time: 12 minutesLanguages: English, SpanishTest Publisher: Wonderlic, Inc.Published: 1937

Reliabilities: ranging from 0.82 – 0.94

Validities:• construct validity studies: see Manual WPT• predictive validity studies: numerous (see Manual WPT).



Personality CharacteristicsInventory

Measurement claim: basic personality dimensionsNumber of tests: 5 scales, or 12 scalesAdministration time: 25 – 30 minutesLanguages: English, SpanishTest Publisher: Wonderlic, Inc.Published:

Reliabilities: ranging from 0.82 – 0.87

Validities:• construct validity studies: see Manual PCI• predictive validity studies: see Manual PCI

SpecificationsSpecifications decision support system decision support system

PROGNOST

PROGNOST is a decision-support system build on an expert system shell, a Bayesian Belief Network system.

Manufacturer (Belief Network Norsys Software Corp. technology and API software): Vancouver, BC. Canada

Network development: CogniMetrics, Inc.San Antonio,TX

Further software development: CogniMetrics, Inc.San Antonio,TX

Further software development includes:• Test manager (administering the tests to applicants; transfers test data to database). • User interface decision support system (available to counselors to read decision outcomes and to add further information).• Report writer

PricingPricing prognostprognost

Software Development $ 22,500• test manager (incl. interface)• user interface decision support system• API programming• report writer

Network Development $ 6,500

These prices hold for the development of a network based on the proposed standard test battery and 5 different job categories to be predicted.

Modifications of the standard system, for example, additional tests, and/or additional job categories are possible.

System development

PricingPricing standard test batterystandard test battery

Two options:

1. Quantity-based License:For example, a license for 100 test administrations of the standard test battery costs: $6,000 + tax. This license can be renewed at any time. Payment in advance.Bulk discounts can be negotiated.

2. Time-limited License:We offer interesting discounts for licenses for a full year of unlimited use of the standard battery.

CogniMetrics, Inc. can provide an estimate of cost savings resulting from implementation of PROGNOST as basis of your selection process:

• direct savings on costs per individual assessment

• indirect savings as a result of increased productivity, or improved success ratios in training programs as a result in improved validity of the prediction.

Tests licensing

Availability Availability

CogniMetrics, Inc. can develop the proposed standard system within three months.

Thanks for reviewing thisCogniMetrics proposal

For further information,Please contact:

Dr. Martin J. Ippel

(210) 481 – [email protected]

CogniMetrics, Inc.CogniMetrics, Inc. 2004 2004

IT Aptitude Battery:

In 2004 CogniMetrics, Inc. published the IT Aptitude Battery, the world’s first battery of fully interactive tests of human intelligence.

The ITAB tests measure ability to deal with procedural knowledge, a crucial aptitude in work environments of the 21th century.

The ITAB tests can be purchased as part of PROGNOST, or separately. For more vendor information, please visit:www.cognimetrics.com, select <products>.http://www.cognimetrics.com

CogniMetrics, Inc.CogniMetrics, Inc. 2003 2003

PROGNOST:

In 2003 CogniMetrics, Inc. developed PROGNOST a method to build customized low-cost decision-support systems for selection decisions.

A decision support system is an expert system that uses all available scientific knowledge to predict success of current or future employees for a particular job or training, based on available information (such as their test scores).

http://www.cognimetrics.com

© cognimetrics, inc., 2005 prognost a web-based personnel assessment system

Documents

webbased tests prognost

content navigator prognost

personnel selection

quality of personnel

userfriendly system

expert system

promotion decisions

available information