agenda for today research statistics communication logic & observation
TRANSCRIPT
Agenda for today
Research
Statistics
Communication
Logic & Observation
SelectQuestion
ReviewResearch
DevelopHypothesis Research
Design
Measurement
Data Analyses CommunicateLogic & Results
Ethics
Another view of where we are
Research Designs
Measurement
Data Analyses
Field Studies
Sample Survey (Meta-analysis)
Scaling
CommunicateLogic & Results
Written Assignments
What we covered so far…what’s next
• Via Methodology Assignment– Submit Nov 2/06
• Via “Rationale of your research” assignment– Submit Dec 14/06
Communicating Logic
What’s Required in the Rationale Assignment?
1. What is the phenomenon?
2. How is it different/similar to another phenomenon?
3. When/where/how is it exhibited? Why?
4. When is it not? Why?
5. What common process explains its occurrence vs. non occurrence?
6. Why might your hypothesis not be supported given you will use valid measures and a large enough sample size?
• Because answering these questions will help you improve your writing skills– Evidence from 2 Meta-analyses: Hillocks, 1985;
Atkinson, 1993
• Because it is similar to what you will do at work when you make an effective proposal or effectively evaluate the outcome of a completed project…– E.g., loan officer proposal
Why do you need to answer these questions?
• Hillocks, 84; cf Atkinson, 1993
Type of instruction G meta-analytic effect
Answering these questions
.56
Grammar -.29
Previous Research shows that answering these questions improves writing quality
• Evaluate loan application from a multi-national that • Owned farms, food processing & manufacturing plants etc. • Defaulted on payment of earlier loan for millions of dollars but submitted a request for
extension on loan & an additional loan
• Bank would consider additional loan to enhance probability of recovering initial loan from multi-national
• While writing, loan officer had to
– Assess performance w/initial loan – Evaluate current viability
• Assess productivity, markets, current financial structure, management, etc.
– Predict potential for recovery & continued viability – Convince superiors recommendation was valid
• Success of recommendation determined officer’s future in bank
Example of One Loan Officer’s Writing Task
(Hillocks, 95)
• Should be clear and narrow in scope– Examples of a hypothesis
• X is related to Y
• Can convert your Focused Research Qs to a hypothesis: – How is x related to y? Why is x related to Y?
– Examples of non-hypotheses• A topic: what is X? what is Y? \
• A research qs: What affects X?
The Phenomenon is Your Hypothesis
• Consider only two variables, not special conditions– What makes it too broad?
• E.g., your research question “what factors affect X” is too broad
• E.g., some focused research questions: “X affects Y, Z, A, B etc” can be too broad
– If it is too narrow you won’t be able to do the rationale assignment
• E.g., x affects y in the presence of z (i.e., finding counter evidence will be hard!)
What makes a hypothesis Good & Narrow?
• Define both variables in hypothesis
1. Give Examples of items of variables2. List features variables3. Explain how both variables in hypothesis are
different from & similar to other, closely-related variables
• Can have either theoretical/empirical sources for definitions of variables in hypothesis– How are definitions similar/different from
methods of measurement?
Differences & Similarities to your Phenomenon: An Overview
1. Give Examples of items
2. Define variables by features– Focus on what the variable is rather than what
are its effects/causes? • E.g., Identify features/dimensions/of job satisfaction
rather than its causes & effects – see lecture notes on Development Index Article for examples
• E.g., Extraversion is – Being socially out-going
– Having a lot of positive affect
2 things to do when defining variables…
3. Compare & contrast variables effectively– E.g., How is job satisfaction different from AND
similar to organizational commitment? NOT how is job satisfaction different & similar to marital satisfaction – poor counter example
– E.g., What is extraversion? How is it different from agreeableness? NOT how is extraversion different from introversion? Introversion is low extraversion!
1 more thing to do when defining variables…
1. List dimensions/features of the variable
2. Give example items/measures that illustrate the variable
3. Identify the contrasting variable that can be confused with examples of the concept
Summary:To define a variable well, you need to…
• Can help you evaluate whether the evidence you found is relevant– More later on relevance
• Can help you understand why there is counter evidence– More later on how you can accomplish this
Why do definitions this way?
What we covered so far
• What is the phenomenon?• Description of 2-variable hypothesis
• How is it different/similar to another phenomenon?• Define BOTH variables• Give effective counter-examples of variables
What’s next
• When/where/how is it exhibited? Why?
• When is it not? Why?
• Exhibition means providing ‘evidence’ which relies heavily on good sources
• What are good empirical sources?– Have reliable & valid facts
• Implications of not having reliable & valid facts to support your counter hypothesis (covered later)
– Logical explanations
Good sources for getting evidence for your hypothesis & counter hypothesis
• At least – 2 good empirical sources to support hypothesis– 1 good empirical source that does not support
your hypothesis • More on what kind of evidence is regarded as NOT
supporting your hypothesis later..
Number of sources needed for your hypothesis & counter hypothesis
• Give specific, relevant & sufficient detail about sources supporting hypothesis & counter hypothesis
• Describe results/methods of source, Focus on relevant findings/methods
• Avoid describing conclusions, describe evidence
– E.g., N? Measures used? Size/direction of correlation? Difference in mean? Significance level? Establish Relevance when describing
Descriptions of sources
• Surveyed 369 managers in 83 business units by using Spreitzer’s 12-item questionnaire for empowerment & Rusbult’s 6-item questionnaire for job satisfaction [and found]....a significant positive correlation [.72] between psychological empowerment and job satisfaction
Example good description Student draft, Spring 2006
• Explaining is different from describing
• Explain why & how empirical findings support pro-hypothesis– Establish relevance of empirical finding to your
hypothesis– Explain why one thing is related to another
• Convert hypothesis into focused research question and answer it
– E.g., extraversion is related to non verbal skills (hypothesis) to WHY is extraversion related to non verbal skills (focused research question)
Why is your hypothesis supported
Extraversion Amount ofSocial experiences
+ NV Skills+
• Extraverts are more skilled at non verbal communication because they have different amounts of social experiences (Student draft Fall 05)
Example explanation of evidence for hypothesis
• Use additional sources to explain findings (e.g., theoretical review) if sources w/good reliable & valid data don’t explain findings well– E.g., Medical Journals
A tip on using sources to explain
What we covered so far..what’s next
• What is the phenomenon?
• How is it different/similar to another phenomenon?
• When/where/how is it exhibited? Why?– Getting good evidence for your hypothesis,
establishing relevance of evidence for your hypothesis, giving a good explanation
• When is it not exhibited?
• Why is not exhibited?
• What is non-support for your hypothesis (i.e., what is your counter hypothesis?)– Directly counters the relation specified in your
hypothesis• E.g., “extraversion is not related to non verbal skills” is
the STRONG counter hypothesis
• “General Ability is related to non verbal skills” is the WEAK counter hypothesis
When is your phenomenon not exhibitedA.K.A. when is your hypothesis not supported
• Don’t use a straw-man type source to support your counter-hypothesis
• Meta-analyses are good sources for getting evidence for your counter-hypothesis– How to find meta-analyses in psycinfo
Good sources for getting evidence for your counter hypothesis
• Describe relevant parts of the evidence you found that – Supports your hypothesis – Counters your hypothesis
• If you didn’t find either yet (as specified in course timetable), set goals on how you will go about finding it– e.g., What terms to use in electronic database? (i.e.,
what are your definitions of your variables?)
Discuss with peer & write down goals to improve
What we covered so far..what’s next
• What is the phenomenon?
• How is it different/similar to another phenomenon?
• When/where/how is it exhibited? Why?
• When is it not exhibited?
• Why is not exhibited?
After describing the reliable counter evidence …e.g., Participation does not lead to setting more difficult goals
1. Discuss explanation for pro-hypothesis finding
2. Demonstrate its limitations• Assumptions, definitions/measurement, nature of
sample etc.
3. State the more valid explanation and illustrate its effectiveness
Explaining the counter evidence
• E.g., Participation does not lead to setting more difficult goals
1. Discuss explanation behind pro-hypothesis• E.g., Participation results in more difficult goals being
set because subordinates want their supervisors to believe that they are highly capable and therefore choose more difficult goals than those that may be assigned to them by the supervisor
C24 Fall05 Student Paper
1st step to explaining the counter evidence
2. Demonstrate limitations of explanations for the pro hypothesis with counter-evidence
• With assumptions made• E.g., the explanation for participation leads to more
difficult goals assumes that supervisors do not know the abilities of the subordinates and so assign easy goals
C24 Fall05 Student Paper
2nd step to explaining the counter evidence
3. State the more empirically valid explanation and illustrate its effectiveness
– E.g., When supervisors know the abilities of subordinates, participation does not result in more difficult goals
C24 Fall05 Student Paper
3rd step to explaining the counter evidence
After describing the reliable counter evidence …e.g., Participation does not lead to setting more difficult goals
1. Discuss explanation for pro-hypothesis finding
2. Demonstrate its limitations• Assumptions, definitions/measurement, nature of
sample, etc.
3. State the more valid explanation and illustrate its effectiveness
Another way to explain the counter evidence
ExtraversionIn Total
Social experiences+ NV Skills+
ExtraversionAs Positive Affect
Social experiences- NV Skills-
Using definitional differences to demonstrate the limitations of pro-hypothesis explanation
AgreeableExtraverts
Social experiences+ NV Skills+
Using nature of sample differences to explain the Counter Evidence
(aka using Moderator Variables)
Disagreeable Extraverts
Social experiences- NV Skills-
• State the more empirically valid explanation and illustrate its effectiveness
– e.g., agreeable extraverts have more social experiences whereas disagreeable extraverts have fewer social experiences
– e.g. when extraversion is measured as frequency of positive affect, it is not related to Non Verbal Skills
C24 Fall05 Student Paper
After using definitional differences or nature of sample differences to
explain the counter evidence
• Explain why & how counter evidence…– Supports the counter hypothesis
• Does it test assumptions made by pro-hypothesis explanations ?
– E.g., Participation does not increase goal difficulty when supervisors know abilities of subordinates
• Does it differ in definitions? Measures used? Size/Nature of sample? Context of study? Etc.?
– Relevance of detail given
Summary of what to do toexplain the counter evidence
• When, where & how is it not exhibited? Why? • Describing evidence against the hypothesis &
explaining why
• What common process explains its occurrence AND non occurrence?
• Why might your hypothesis not be supported given you will use valid measures and a large enough sample size?
What we covered so far…what’s next
• Differences must be explained by common explanation– Can identify differences in studies & relate to
meaningful variable within phenomenon• e.g., agreeable extraverts have more social experiences
whereas disagreeable extraverts have fewer social experiences
• e.g. when extraversion is measured as frequency of positive affect, it is not related to Non Verbal Skills
What explains the differences in evidence for & against hypothesis?
• Explaining valid counter evidence should lead you to answer how the two pieces of evidence can be reconciled with current/future research=creative insight & good analytical skills!
Why explain the differences for & against hypothesis using valid evidence?
• How & why does the evidence you found so far– Supports your hypothesis – Counters your hypothesis
• NOTE: describing is different from explaining
• If you don’t know the answer set goals on how you will go about answering this question
• Why might your hypothesis not be supported given you have reliable & valid measures and a large enough sample size
Discuss with peer & write down goals to improve
1. What is the phenomenon? • Description of 2-variable hypothesis
2. How is it different & similar to another phenomenon?
• Definition of variables
3. When, where & how is it exhibited? Why?• Describing evidence for the hypothesis &
explaining why
Summary of what is involved in “rationale assignment”
4. When, where & how is it not exhibited? Why?
• Describing evidence against the hypothesis & explaining why
5. What common process explains its occurrence vs. non occurrence?
• Why is there a difference in the findings?
6. Why might your hypothesis not be supported given you will use valid measures and a large enough sample size?
Summary of what is involved in ” rationale assignment” (cont’d)
Research Designs
Measurement
Data Analyses
Field Studies
Sample Survey (Meta-analysis)
Scaling
CommunicateLogic & Results
Written Assignments
What we covered so far…what’s next