leverhulme methods presentation
TRANSCRIPT
Leverhulme 1
Overview of different Overview of different Qualitative Analysis Qualitative Analysis
MethodsMethods
Dr. Anne [email protected]
Leverhulme 2
OVERVIEWOVERVIEW• Paradigm background (research onion)• Analysis Methods
– Content Analysis – Discourse Analysis– Thematic Analysis– Grounded Theory– Phenomenological Analysis– Research Quality and Action Research
Leverhulme 3
RESEARCH PARADIGM
Leverhulme 4
Analysis paradigm
• Crotty (1998) rigorous = identify / defend research strategy decisions
• Adams et al (2008) rigour from appropriate identification and application of methods according to research questions
• Reflexivity essential to maintain research rigour
Qualititative methodsQualititative methods
• In-depth Interviews,
• Focus groups,
• Observational / ethnographic studies,
• Open ended data (system logs)
• Data transcribed
QUANT / QUAL ComparisonQuantitative approaches Qualitative approaches
'Simple' numeric data 'Complex' rich dataMeasurement Meaning
Explanation UnderstandingPrediction Interpretation
Generalisable account Contextual accountRepresentative population sample Purposive/ representative
perspective sampleHypothesis-testing ExploratoryClaims objectivity Accepts subjectivity
Closed system (experimental control)
Open system(ecological validity)
Qualitative / Quantitative DivideQualitative / Quantitative Divide
Research is like fishing
Quantitative methods
• You find the best river for the fish you want, you have one line, a specific bait for a specific type of fish.
Qualitative methods • You may want to catch tuna so you fish in certain parts of the sea BUT on the whole you throw your nets out to sea and catch everything including the things you want and don’t want.
Qualitative Analysis MethodsQualitative Analysis Methods
• Conversational / Discourse Analysis• Thematic Analysis / Grounded Theory • Content Analysis / Critical Incident Analysis
• counting• imposing established frameworks
“Both qualitative and quantitative approaches share a common concern with theory as the goal of research” (Henwood & Pidgeon, 1992 p.101)
Leverhulme 9
Quantitative methodsQuantitative methods
• Experiments
• Questionnaires
• System logs (producing data for statistical analysis)
Leverhulme 10
Research Paradigms:Research Paradigms:QuantitativeQuantitative
• Imposes prior theories (discovery)
• Reliant on hypothetico-deductive method to establish causal relationship (justification)o Operationalised (reductionistic)
o Measured
Leverhulme 11
Research Paradigms:Research Paradigms:Quantitative LimitationsQuantitative Limitations
• Validityo Inappropriate fixing of meaningso Imposing external system of meaning
for internal subjective structures• Complexity of data lost through
reductionistic approach
Leverhulme 12
Qualititative methodsQualititative methods
• In-depth Interviews
• Focus groups
• Observational / studies
• Open ended data
Leverhulme 13
Research Paradigms:Research Paradigms:QualitativeQualitative
• Generates working hypothesis by producing concepts from data
• Represents participants reality in its complex context
Leverhulme 14
Research Paradigms:Research Paradigms:Qualitative LimitationsQualitative Limitations
• Subjectivityo Data collection procedures
o Analysis
• Reliabilityo Across context & researcher
Leverhulme 15
The research onion: TEL / HCI
Leverhulme 16
TEL example : Juxtalearn
• An Educational design project in field of TEL • HCI perspective connects computing and education • ‘in the wild’ practice based perspective to TEL see Adams et al
(2013, 2014) expanded in Fitgerald et al (2015)• Research onion simplified - Four main research epistemologies
& related methods: overlay of TEL & HCI• Juxtalearn inspired by action research and design-based
approaches from a ‘pragmatic’ perspective. • But used both qualitative and quantitative analysis methods to
mixed method data collection. Grounded Theory Analysis.
Leverhulme 17
Content Analysis
• Quantitative analysis of qualitative data
• Counting instances
• Either using pre-existing coding framework or creating one (problems with both).
• Careful to apply this it needs linked to other analysis methods (neither quant nor qual)
Leverhulme 18
Content Analysis Coding
• Take data• Categorise data – often with pre-determined
models, frameworks and coding schemes• Code the data according to category• Count the data – instances of the data
Leverhulme 19
Example of coding Units
Unit ExamplesWord social engagement wordsTheme occasions of social engagementItem whole accounts of SECharacter different characters roles in SETime & Space space & time-logs in blog spent
on SE activities
Leverhulme 20
Discourse Analysis• Very different from other forms of QA• In pure form: based on meaning and language
used – for that ‘discourse’• From a Radical constructivist approach (e.g.
discourse action model)• Memories not facts but socially
reconstructions• Pure: naturally occurring talk• Need to fully understand philosophy to use it
Leverhulme 21
Discourse Analysis Coding
• TEL: specific impact of technology on the discourse e.g. virtual reality impact on learning speech
• Code every utterance, gaps, breaths• Identify language usage: plays, stock
phrases, how language is used• Not understanding of the concepts generally
Leverhulme 22
Sections of code4. [J.H.] you are black or Asian you cannot join (.) this national (.) political party5. (.) slightly different I think from the metropolitan police black6. Officers association wouldn’t you say?7. (.)8. [N.G.] I don’t think it’s different in the slightest because as the simple fact9. is as I say that white police officers couldn’t set up an association10. Of their own er if for instance they feel they’re being Overlooked for11. ap-prom for promotion so every different ethic group er in 12. this (.) multicultural multiethic society that our
Coolican, 2014
Leverhulme 23
Thematic Analysis
• Data gathering and analysis approach• Understanding drawn from the data• Recognise limitations but not based on
specific epistemology (many data types)• Can be used to provide generic
understanding but limited application to theory building
Leverhulme 24
Thematic Analysis Coding
• 1- familiarise with data• 2 – generatie initial codes• 3 – search for themes• 4 – review themes • 5 – define and name themes
Leverhulme 25
Example of Blog code
Coolican, 2014
Leverhulme 26
Grounded Theory
• Analysis grounded in the data• Mixed methods data – quant and qual• 1st stages similar to TA• Systematic merging of mixed data • Development of a ‘theory’ (also models)• Process pushes analysis of rouge / awkward
data to reveal ‘novel findings’
Leverhulme 27
Grounded Theory Coding
• Open coding – line by line (as with TA)• Memo writing – capturing insights for
theory• Axial coding – linkages between categories• Selective Coding – important themes ‘high-
level storyline. • Process effects – conditions, actins, effects
effect relationships in a process
Leverhulme 28
Example process effect model
4 1
3
IS
TRUST privacy secured
based on assumptions
Users
IR
Contexts
IU
Technology & its implementation make assumptions inaccurate
Increased perceived privacy invasions
2
Decreased trust in organisationEmotive reaction Reject technology
Leverhulme 29
Phenomenological AnalysisInterpretive Phenomenological Analysis (IPA)
• The individuals ‘experience’ events• Individuals perspective on and understanding
in the world. • Understands constructivist role of researcher
in interpreting individual experience.• Unlike DA takes what people say as realistic
account (semi-structured interviews)
Leverhulme 30
IPA coding
• Read text ‘eyeball it’ • Comments on transcripts (varied in size and
length)• Interpretations / analysists reactions /
recurrent points / queries about meaning• Identify themes (hierarchical themes)• Compare and contrast themes
Good Quality Research
• Henwood / Pidgeon (1992)– good quality research
• 7 golden rules of good quality research
• Exploratory (discovery) – reductionistic (justification)
• Not Divide but to compliment
• ISSUES OF BIAS
Levels in research
PLAN: Background research & planning
ACT and /or OBSERVE
REFLECT: Implications / Discussion
Initial QUESTION?
Action Research & Levels
PLAN: Background research & planning
ACT and /or OBSERVE
REFLECT: Implications / Discussion
Initial QUESTION?
PLANNING courses, teaching activities
REFLECT on Changes to current practices
CONDUCTING EVERYDAY ACTIONS (E.G. Teaching)
Action Research & Exploratory Research
PLAN
REFLECT ACTACT
OBSERVE
What is my question?
Action Research (2)
Plan
Reflect Act
Observe
Initial Question
Plan
Reflect Act
Observe
Further Question
Leverhulme 36
References
Adams, A., FitzGerald, E. and Priestnall, G. (2013). Of catwalk technologies and boundary creatures. ACM Transactions of Computer-Human Interaction, 20(3), article no. 15. download pre-print from: http://oro.open.ac.uk/35323/ Adams, Anne and Cox, Anna L. (2008). Questionnaires, in-depth interviews and focus groups. In: Cairns, Paul and Cox, Anna L. eds. Research Methods for Human Computer Interaction. Cambridge, UK: Cambridge University Press, pp. 17–34. download from : http://oro.open.ac.uk/11909/Adams, Anne; Lunt, Peter and Cairns, Paul (2008). A qualititative approach to HCI research. In: Cairns, Paul and Cox, Anna eds. Research Methods for Human-Computer Interaction. Cambridge, UK: Cambridge University Press, pp. 138–157. download from : http://oro.open.ac.uk/11911/ Coolican, H (2014) Research methods and statistics in psychology. Psychology press, NY. Crotty, M.J. (1998) The Foundations of Social Research: Meaning and Perspective in the Research Process. Thousand Oaks, CA: Sage.FitzGerald, E. and Adams, A. (2015). Revolutionary and evolutionary technology design processes in location-based interactions. International Journal of Mobile Human Computer Interaction, 7(1) pp. 59–78. download from : http://oro.open.ac.uk/42067/ Henwood, K. L., & Pidgeon, N. F. (1992): “Qualitative research and psychological theorising.” British Journal of Psychology. 83, 1, pp. 97-111.Saunders, M., Lewis, P. and Thornhill, A. (2012) Research Methods for Business Students, 6th edition. Harlow: Pearson Education Ltd