crafting and validation of instrument
TRANSCRIPT
Construction and Validation of Research
Instrument
Flerida S. PajarillagaEPS II- HRD
By the end of this seminar-workshop, the participants should be able to: Understand the usual steps in
crafting a data-gathering tool Be able to apply the principles in
question construction Conduct basic validation of created
instrument for action research
Instrument Design Designing an
instrument is challenging and a complex process.
Consider whether your variables are measurable
Consider modifying an existing instrument
TYPICAL STEPS IN INSTRUMENT DESIGN
Write different types of questionsUse Strategies for
Good Question construction
1
2
Perform a pilot test of the questions
3
Write Different Types of Questions
1
Personal, Attitudinal and Behavioral Questions
Sensitive Questions Open-and Close-
Ended Questions
Personal, Attitudinal and Behavioral Questions
Three popular Types
Other possible descriptors include:
• Exemplary, Proficient, Marginal, Unacceptable
• Advanced, High, Intermediate, Novice
• Beginning, Developing, Accomplished, Exemplary
• Outstanding, Good, Satisfactory, Unsatisfactory
Personal, Attitudinal and Behavioral Questions
Three popular Types
Behavior Did you take a semester off during any of your four years in college?______ Yes______ No
Sensitive Questions This is usually written when the respondent has “warmed up” answering neutral questions Place leading questions like: Instead of :
Have you taken drugs?
Ask:In past surveys, many men reported at some point in their lives they have taken marijuana. This could happen during adolescence or as an adult. Have you ever smoked Marijuana?
Open- and Closed-Ended Questions
CLOSE ENDED QUESTIONSThe researcher poses a question
and provides preset responses or optionsEXAMPLE:There are many reasons why adults wish to get more education. What is your most important reason for coming to adult basic education classes (Check one.)___________ To be able to help my children with their schoolwork___________ To get a better job___________ To improve myself___________ To get a high school equivalent diploma
Open- and Closed-Ended Questions
OPEN ENDED QUESTIONSquestions for which researchers do
not provide options, participants provide responses
EXAMPLE:
Why are you attending adult education classes?____________________________________________________________________________________________________________
Open- and Closed-Ended Questions
SEMI-CLOSED-ENDED QUESTIONShas the advantage of open-and close-ended questions
EXAMPLE:There are many reasons why adults wish to get more education. What is your most important reason for coming to adult basic education classes (Check one.)___________ To be able to help my children with their schoolwork___________ To get a better job___________ To improve myself___________ To get a high school equivalent diploma___________ Other (please comment) _______________________
Use Strategies for Good Question construction
2
Pay attention to the quality of your questions
It should be clear/ unambiguous that it does not confuse participants Consider cultural background and avoid gender-biased words
Eliminate the problems
Problem no. 1:
Do you support gun control?
_____ YES_____ NO_____ Don’t Know
Do you believe that guns do not belong in school?
_____ YES_____ NO_____ Don’t Know
UNCLEAR QUESTIONS
Problem no. 2:
Do you believe that guns and knives do not belong in school ?
_____ YES_____ NO_____ Don’t Know
Do you believe that knives do not belong in school?
_____ YES_____ NO_____ Don’t Know
MULTIPLE QUESTIONS
Problem no. 3:
Whenever violence occurs in schools, weapons are typically found in school lockers. Do you believe that students should have gun in the lockers?
_____ YES_____ NO_____ Don’t Know
Should students have guns in their lockers?
_____ YES_____ NO_____ Don’t Know
WORDY QUESTIONS
Problem no. 4:
Students should not carry weapons and not have them in their lockers. Do you Agree?
_____ Strongly Agree_____ Agree_____ Undecided_____ Disagree_____ Strongly Disagree
Should students have guns in their lockers?
_____ YES_____ NO_____ Don’t Know
NEGATIVELY WORDED QUESTIONS
Problem no. 5:
Should students pack a .45 at school?
_____ YES_____ NO_____ Don’t Know
Should students have handgun in their lockers?
_____ YES_____ NO_____ Don’t Know
QUESTIONS WITH JARGON
Problem no. 6:
How many times have you seen a student carry a handgun?
_____ 0 times_____ 1-2 times_____ 2-3 times_____ More than 3
times
How many times have you seen a student carry a handgun?
_____ 0 times_____ 1-2 times_____ 3-4 times_____ More than 4
times
OVERLAPPING RESPONSES
Problem no. 7:
To what extent do you feel that handguns are a problem at your school
_______ A great extent_______ Some_______ Not very
important_______ Not a problem at all
To what extent do you feel that handguns are a problem at your school
_______ A great extent_______ Some extent_______ Little extent
UNBALANCED RESPONSE OPTION
Problem no. 8:
How often have you seen students carry a semi-automatic weapon?
_____ None _____ 1 time_____ 2 times _____ 3 or more times
How many times have you seen a student carry a riffle at school?
_____ None _____ 1 time_____ 2 times _____ 3 or more times
MISMATCH OF THE QUESTION AND THE ANSWERS
Example no.1
Problem no. 8:
How many students have you seen carrying guns at school?
_____ 1 student_____ 2 students _____ 3 students______ More than 3
students
MISMATCH OF THE QUESTION AND THE ANSWERS
Example no.2
Have you seen students carrying guns at school?
_____ Yes _____ NoIf Yes, how many
students?_____ 1 student_____ 2 students _____ 3 students______ More than 3 students
Problem no. 9 & 10:
OVERLY TECHNICAL LANGUAGE
QUESTIONS NOT APPLICABLE TO ALL PARTICIPANTS
Example:What would be the appropriate care for children with Progeria?
What database would be most user-friendly for school-base paperless database system?
____FoxPro _____ Lotus123 _____Access
Pilot Test Questions 3
Pilot test is a procedure in which the researcher makes changes in an instrument based on feedback from small number of participants who evaluated the questions. The participants makes comments while taking the test and the researcher makes changes
Exclude this participants on the final sample of the study
ARE YOUR INSTRUMENTS RELIABLE
AND VALID?
Individual scores should be nearly the same or stable over repeated administration of the instrument and should be free from measurement errors
RELIABILITY
Types Reliability : Test re-test, Alternate Forms, Interrater and the split half
The intended test interpretation of the concept or construct that the test assumes to measure matches the proposed purpose.
VALIDITY
Types of Validity: Criterion-related (predictive and concurrent) validity, Content validity and construct validity
STEPS IN CONSTRUCTING A STANDARDIZED INSTRUMENT (Benson & Clark, 1983)
PHASE 1Planning
PHASE 2Construction
Identify purpose of test and target groups
Define Domains of the Test Review literature on construct/
variable Select item format Develop Table of Specifications Validate content – have judges
complete qualitative evaluation Revise Items
PHASE 3Quantitative Evaluation
PHASE 4Validation
Pilot Test Calculate reliability and item
analysis Revise Items
Administer second pilot test Run item analysis Repeat steps of revision Validate (until satisfied!)
STEPS IN CONSTRUCTING AN INSTRUMENT (Benson & Clark, 1983)
LET’S CREATE OUR TOOL!
WORKSHOP1. Participants will be divided into
groups according to research theme assigned
2. Select a type of tool and construct the instrument (at least 5-10 items) with the following details:
(a) Research Title/ Question (b) Participants of the Study (c) Question Items
WORKSHOP
4. Each group will have a partner who will critique (face /content validate ) their work
5. Revise the items
School Principals’ Influence on Trust
.
RESEARCH STUDY No. 1
Reference:Sheldon, D.L, Angell, M.E., Stoner, J.B. & Roseland, B.D., (2010) . School Principals’ Influence on Trust: Perspective of Mothers of Children with Disabilities. Journal of Educational Research 103, 159-170
Trust Worthy Leadership Matrix(Mishra, 1996; Tschannen-Moran & Hoy, 1998, 2000)
Trustworthiness is the willingness to be vulnerable to another on the basis that the other is benevolent, honest, open, reliable and competent
Trust Worthy Leadership Matrix(Mishra, 1996; Tschannen-Moran & Hoy, 1998, 2000)
5 Functions of a LeaderVisioning creating a trustworthy environmentModeling Continuity between word and
example. Lead quietly, exercising restraint and modesty
Coaching assist people to move forward through conversation - know when to push and to back off.
Managing delegate control, high-trust= fewer rules, Trust with parents = less time explaining actions.
Mediating Dealing with betrayal and conflict skillful in conflict management
Sample: Table of Specification
Visioning Modelling
Coaching
Managing Mediating
I encourage my team to participate when it comes decision making time and I try to implement their ideas and suggestions.
I will be the first one to follow the rules set.
I enjoy coaching people on new tasks and policies
Nothing is more important than accomplishing a goal or task.
When correcting mistakes, I do not worry about jeopardizing relationships.