TOOLS/INSTRUMENTS OF

RESEARCH

1. CAHYA ADY P. (10420184)2. DWI PRASETYO A. (10420192)3. TAUFIK SETIAWAN (10420209)4. ANDRIE PRASETYO (10420225)5. TRI AGUS SETIAWAN (10420380)

1. QUESTIONNAIRE2. OPINIONNAIRE/ATTITUDE SCALE3. TEST

Kind of tools of research

A technique for obtaining information from subjects.

A series of question ask to individuals to obtain statistically useful information about a given topic.

Questionnaire?

relativelly economical

has the same question for all subject

can ensure anonimity

Why must be Questionnaire?

How to make Questionnaire

Requirements

Determine the type of information

Limit responses

Figure out who, what, when and where you're going to distribute your questionnaire.

Straigh forward& clear

Quantity

State introduction& purpose

Simple, short & in a page

Double check

Profesional, easy read, & understandable

A. Decision about question contentB. Decision about question wordingC. Decision about form of response to the

questionD. Decision about the place of the question in

the sequence

Guide for constructing questionnaire

types of questionnaire

Types Types

Closed-ended

Open- ended

Contingency

Matrix

No option answer/predefined categories are suggested.

A question that is answered only if respondent gives a particular response to a previous question.

identical response are assignned to multipe questions.

Respondents’ answers are limited to a fixed set of responses.

Provide for marking a yes or no, a short response, or

checking an item from a list of suggested responses. Why did you choose your graduate work at this

University?

Rank:

a. Advice of friend.

b. Scholarship aid.

c. Reputation of University.

d. other

CLOSED-ENDED

Completely unstructured, free response in the respondent’s own word, no clues are given.

Why did you choose your graduate work at this University?Answer :_________________________________________________________________________________________________________

OPEN-ENDED

Differences

Closed-Endeda. highly structuredb. Generate frequencies of responsec. Amenable to statistical treatment &analysisd. More focuse. To the point

Open- endeda. Unstructuredb. Suit for complex issuec. respondents can explore more their thoughtsd. Not limited of pre-set categoriese. Lead to irrelevant & redudant information

chracteristics

•Who are interviewed and their background /experiences.

Factual (Demographi

c)

•What they do, or did in the past.

Behavioural

•Attitudes, opinions, beliefs, interests, and values.

Attitudinal

Types Gained Information of Questionnaire (dornyei, 2003, PP.8-9)

1. Significant topic2. Single source question3. Short, simple and understandable4. Attractive in appearance, neat arrangement

and clearly duplicated5. clear and complete direction, simple and clear

questions, and single idea or concept 6. good psychological order and it proceeds

general to more specific responses7. objective questions8. easy to tabulate and interpret

Characteristics of a Good Questionnaire

The question of content validity is: “Do the items sample a significant aspect of the purpose of the investigation?”

Clearly defined meaning in

questionnaires

Get help from colleagues or experts from the field

of questions

The panel of experts can rate the instrument to know how

effectively it samples significant aspects of its purpose

Estimating the predictive validity of a questionnaire by a follow-up

observation of respondent behavior at the present time or

at some time in the future

Validity and reliability of questionnaires

The information form that attempts to measure the attitude or belief of an individual.

How people feel, or what they believe is their attitude. But it’s difficult, if not impossible, to describe and measure attitude. Researchers must depend upon what people say are their beliefs and feelings. This is the area of opinion. Through the use of questions, or by getting people’s expressed reaction to statement, a sample of their opinions is obtained.

OPINIONNAIRE/ATTITUDE SCALE

1. Thurstone Technique

( equal – appearing Interval Scales)

2. Likert Technique

( Summated Rating

Scale )

Types of Attitude Scale

The list of statement is given to the subjects, who are asked to check the statements with which they agree. The median value of the statements that they check establishes their score, or quantifies their opinion

1. Thurstone Technique

The second method, the Likert Method of Summated Ratings, which can be carried out without the panel of judges, has yielded scores very similar to those obtained by the Thrustone method. The coefisient correlation between the two scales was reported as +.92 in one study. Since the Likert Scale takes less time to construct, it offers an interesting possibility for the student of opinion research.

The Likert scaling technique assigns a scale value to each of the five responses. Thus, the instrument yields a total score for each respondent and a discussion af each individual item, while possible, is not necessary.

2.Likert Technique ( Summated Rating Scales)

a. Strongly agree 5

b. Agree 4

c. Undecided3

d. Disagree2

e. Strongly disagree1

Starting with a particular point of view, all statements favoring this position would be scored:

Scale value

For statements opposing this point of view, the items are scoresIn the opposite order:

Scale valuea. Strongly agree 1b. Agree 2c. Undecided 3d. Disagree 4e. Strongly disagree 5

1. Heaven doesn’t exist as an actual place or location

2. God sometimes sets aside natural law, performing miracles

3. Hell does not exist4. The devil exists as an actual

person5. God is a cosmic force, rather

than an actual person6. There is a final day judgment for

all who have lived on the earth.

ilustrationa b c d e

a b c d e

a b c d e

a b c d e

a b c d e

a b c d e

If the opinionnaire consisted of 30 statements or items, the following score value would be revealing:

30 x 5 = 150 Most favorable response possible

30 x 3 = 90 A neutral attitude30 x 1 = 30 Most unfavorable attitude

The scores for any individual would fall between 30 and 150; above 90, if opinions tended to be favorable,below 90, if opinions tended to be unfavorable to the the given point of view.

Definition :

a method of measuring a person’s ability, knowledge, or performance in a given domain.

TEST

1. ACHIEVEMENT TEST:attempt to measure what an individual has learned – his or her present level of performance. Most test used in schools are achievement tests. Achievement test scores are used in placing, advancing, or retaining students at particular grade level. Achievement tests scores used to evaluate the influence of course study, teachers, teaching methods, etc/

TYPE OF TESTS

2. The Proficiency TestThe proficiency test also measures what students have learned, but the aim of the proficiency test is to determine whether this language ability corresponds to specific language requirements.

There are some samples of proficiency tests:  1. Placements test

The student proficient enough to enter intermediate course or it is better to place him or her in basic course.

2. The reading- knowledge tests for doctoral candidates

Example: Is the student able to read professional literature in another language with a specific level of accuracy?

The progress test measures how much the student has learned a specific course of instruction. The tests that the classroom teacher prepares for administration at the end of a unit or the end of a semester are progress tests.

The Progress Test

Multiple choiceArranging sentenceFill in the blankMatchingTrue or falseShort answer

Types of test items

Constructing multiple- choice items

1. Design each item to measure a specific objectives

2. State both stem and option as simply and directly as possible

3. Make certain that intended answer is clearly the only correct one

4. Use item indices to accept, discard, or revise items.

Analysis ofMultiple-Choice Tests

1. To develop the quality of test items through the students’ answers for each item.

2. To find out whether the test items will be accepted, discarded, or revised.

1.IF (item facility or item difficulty)2.ID (item discrimination and sometimes called

item differentiation) 3.Distractor

How to analyze?

IF (item facility)

The extent to which an item is easy or difficult for the proposed group of test-takers.

Brown (2000) defines item facility as the proportion of students who answered a particular item correctly.

The proportion is shown in index of IF.

The formula used to get the item facility

The number of test-takers answering the item correctly

IF ----------------------------------------------------------------------------- The number of test-takers

e.g. if you have an item on which 13 out of 20 test-takers respond correctly, your IF index is:13 divided by 20 equals .65 (65%)

The Classification of IF Item

0.00—0.30 The item is difficult0.31—0.70 The item is medium0.71—1.00 The item is easy

The extent to which an item differentiates between high and low ability test takers. An item on which high ability students and low ability students score equally well would have poor ID because it didn’t discriminate between the two groups.

ITEM DISCRIMINATION (ID)

The formula for calculating ID is

High group # correct – low group # correct ---------------------------------------------------

½ x total of your two comparison groups

Item #23 # Correct # Incorrect

High-ability Ss (to 10) 7 3

Low-ability Ss (bottom 10) 2 8

Using the ID formula (7-2 = 5 ÷ 10= .50), you would find that this item has an ID of .50, or a moderate level.

The Classification of ID Item

0.40—1.00 The item is accepted0.30—0.39The item is accepted but it needs

revising0.20—0.29The item must be revised0.19—0.00The item is discarded

(Safari; 2005: 27)

The functions of ID are:

To increase the quality of an item of test through the empirical data. Based on the index of the ID, each item will be found out to be good, be revised, or be discarded.

To find out how far each item can detect and distinguish the students’ ability, that is students who understand or do not understand the material taught by the teachers.

Is one more important measure of multiple choice item’s value in a test, and one that is related to Item Discrimination (ID)

The pattern is obtained by calculating the number of test-takers who choose the option of test item or who do not choose any options. The distribution pattern will show whether or not the distractor works well.

DISTRACTOR

Example:

Choices A B C D E

High-ability Ss (10) 0 1 7 0 2

Low-ability Ss (10) 3 5 2 0 0

Note: C is the correct response

The characteristics of distractor is:

the distractors should be brief and as homogeneous as possible,

the distractors should be plausible, use at least 3 distractors to reduce the chance of

guessing the correct answer,avoid distractors that provide clues, the distractors should embody misconceptions,

partly correct answers and common errors of fact or reasoning.

TEST LAYOUTClear Instruction

What to do Appropriate response

Time allowance How many items to attempt

Example:

Test consists of 20 questions. Read each question carefully. In the button beside each answer, check the

letter that indicates the correct answer for each question. 20 minutes are given to complete the test.

TEST TIMINGIt will determine how ready, motivated, and

capable a student might be.The best time is between 11 a.m. to 1

p.m.Time allowance should be told to the

students.One should consider the level of difficulty in

setting out the time allowance.

SCORINGFigure out the objectives and determining the

weight of each item.

Percent of

Total Grade

Possible Total

Correct

Listening 30% 20 items @ 1.5 points each = 30

Grammar 40% 20 items @ 2 points each = 40

Reading 30% 20 items @ 1.5 points each = 30

Total 100