Qualitative Data Qualitative Data Analysis: An introductionAnalysis: An introduction

Carol GrbichCarol Grbich

Chapter 15: Chapter 15:

Content AnalysisContent Analysis

Content analysisContent analysis When to useWhen to use: when you have large sets of existing written or visual : when you have large sets of existing written or visual

documentation for analysisdocumentation for analysis

Type of question best suitedType of question best suited: : What is the percentage of occurrences of ‘X’ words, events, types of What is the percentage of occurrences of ‘X’ words, events, types of approaches etc. approaches etc. How have particular concepts been used in context and why? and How have particular concepts been used in context and why? and for what purpose?for what purpose?

StrengthsStrengths: enumerative provides a numbers oriented overview, : enumerative provides a numbers oriented overview, while ethnographic provides thematic analysis with more depth while ethnographic provides thematic analysis with more depth of explanation as to why and how words have been used in of explanation as to why and how words have been used in particular cultural contexts.particular cultural contexts.

WeaknessesWeaknesses: Enumerative data alone provides only a superficial : Enumerative data alone provides only a superficial overview and thematic contextual interpretation alone lacks the overview and thematic contextual interpretation alone lacks the detailed numerical information to situate and structure the data.detailed numerical information to situate and structure the data.

Content analysis : Process 1.Content analysis : Process 1.

1.1. Do you have sufficient Do you have sufficient documentsdocuments to make this form of to make this form of analysis useful? which aspects of these documents are to analysis useful? which aspects of these documents are to be analyzed? All of the documents? part of the be analyzed? All of the documents? part of the documents? and pertaining to what topics?documents? and pertaining to what topics?

2.2. What What sampling sampling approach will be undertaken? random, approach will be undertaken? random, stratified, cluster or non probability approaches?stratified, cluster or non probability approaches?

3.3. What What level of analysislevel of analysis will be undertaken and what will be undertaken and what particular concepts or situations will be coded for? particular concepts or situations will be coded for? How will you incorporate any thematically analysed data: How will you incorporate any thematically analysed data:

as a basis for the generation of codes? as a basis for the generation of codes? as a basis for cross checking? to identify discourses? as a basis for cross checking? to identify discourses? or to provide depth information and case studies?or to provide depth information and case studies?

Content analysis : Process 2Content analysis : Process 2

4.4. How will the protocol and/or your codes be generated? How will the protocol and/or your codes be generated? via preliminary data and thematic analysisvia preliminary data and thematic analysis? ?

via a predecided (a priori) coding frame derived from the via a predecided (a priori) coding frame derived from the literature and your own experiences of this field? literature and your own experiences of this field?

5. Will you look at context? or stay with a broad numerical 5. Will you look at context? or stay with a broad numerical overview?overview?

6. How reliable is the approach or protocol that you have decided 6. How reliable is the approach or protocol that you have decided on? on?

Can a high level of inter-coder reliablilty be sustained? Can a high level of inter-coder reliablilty be sustained? Can validity be achieved through cross referencing to Can validity be achieved through cross referencing to

other documents other documents or through triangulation and the inclusion of or through triangulation and the inclusion of

qualitative qualitative data? data?

Enumerative content analysis toolsEnumerative content analysis tools Word frequencyWord frequency - helps you to - helps you to identify how often key identify how often key

words are turning up in your documents.words are turning up in your documents. Key word in context or concordanceKey word in context or concordance. This approach shows . This approach shows

each word in the document in alphabetical order and in each word in the document in alphabetical order and in context. context.

Category frequency or cluster analysis Category frequency or cluster analysis wherewhere other related other related words (synonyms) will also be picked up, words (synonyms) will also be picked up,

LemmatizationLemmatization where the base form of the word and its where the base form of the word and its variations are gathered variations are gathered

co-occurrenceco-occurrence of particular words such as ‘security’ and of particular words such as ‘security’ and ‘terrorism’‘terrorism’

Word frequency: WordleWord frequency: Wordle

//

http://www.spudart.org/blogs/randomthoughts_comments/4758_0_3_0_Chttp://www.spudart.org/blogs/randomthoughts_comments/4758_0_3_0_C

Key word in context TACTwebKey word in context TACTweb

Database Title: A Midsummer's Night Dream Database Title: A Midsummer's Night Dream Query: love Query: love love (5/99)love (5/99) [Exit PHILOSTRATE][Exit PHILOSTRATE] Hippolyta, I woo'd thee with my sword,Hippolyta, I woo'd thee with my sword, And won thy And won thy lovelove,, doing thee injuries; doing thee injuries; But I will wed thee in another key,But I will wed thee in another key, With pomp, with triumph and with With pomp, with triumph and with

revelling.revelling. ------------------------------------------------------------- -------------------------------------------------------------

I.1/577.1I.1/577.1

Cohen’s Cohen’s KappaKappa

Inter coder reliabilityInter coder reliabilityPPAA is the proportion of units on which the raters agree is the proportion of units on which the raters agreePPcc is the proportion of units for which agreement is is the proportion of units for which agreement is expected by chanceexpected by chance

0.00 = poor agreement0.00 = poor agreement0.21 – 0.40 = fair agreement 0.21 – 0.40 = fair agreement 0.81 – 1.00 = high agreement0.81 – 1.00 = high agreement

http://www.kokemus.kokugo.juen.ac.jp/service/kappa-e.html

Ethnographic content analysisEthnographic content analysis

The basic steps include:The basic steps include: Location of all relevant documents - sample if Location of all relevant documents - sample if

desirable.desirable. Identification of the units to be analysedIdentification of the units to be analysed Development and testing of a protocol from the Development and testing of a protocol from the

intensive analysis of a few documents intensive analysis of a few documents Revision and further refinement of the protocol as Revision and further refinement of the protocol as

analysis proceedsanalysis proceeds Interpretation of meaning within content and Interpretation of meaning within content and

cultureculture

Content analysis: AdvantagesContent analysis: Advantages

Can simplify very large documents into Can simplify very large documents into ennumerative informationennumerative information

Can analyse interactions from a distance providing Can analyse interactions from a distance providing a sense of ‘objectivity’a sense of ‘objectivity’

Can identify intentions, attitudes and emotions as Can identify intentions, attitudes and emotions as well as reveal lines of propoganda, inequality and well as reveal lines of propoganda, inequality and powerpower

Can combine both qualitative and ennumerative Can combine both qualitative and ennumerative approaches to look at relationships among approaches to look at relationships among numbers and relationships between these and the numbers and relationships between these and the cultural context.cultural context.

Content Analysis:Content Analysis: DisadvantagesDisadvantages

Can be criticised for being too positivist in orientation Can be criticised for being too positivist in orientation particularly when only enumerative approaches are usedparticularly when only enumerative approaches are used

Limited or poor sampling strategies can lead to biasLimited or poor sampling strategies can lead to bias

Interpretations of words may be limited by the dictionary Interpretations of words may be limited by the dictionary capacity of the computer programcapacity of the computer program

Can decontextualise information; converting texts into Can decontextualise information; converting texts into categorical variables constructs a limited frame in terms of categorical variables constructs a limited frame in terms of interpretationinterpretation

Can be a-theoretical with minimalCan be a-theoretical with minimal interpretation on the interpretation on the assumption that numbers say it allassumption that numbers say it all