Content AnalysisTHE PSYCHOLOGY OF AUDIENCE ENGAGEMENT

Dr. Pamela Rutledge | prutledge@fielding.eduDr. Jerri Lynn Hogg | jhogg@fielding.edu

Qualitative Research

A form of social inquiry that focuses

on the way people interpret and

make sense of their experiences

and the world in which they live. ““

Main Types of Qualitative Research

Case Study

Attempts to shed light on a phenomena by studying in depth a single case example of the phenomena.  The case can be an individual person, an event, a group, or an institution.

Grounded Theory

Theory is developed inductively from a corpus of data acquired by a participant-observer.

Phenomenology

Describes the structures of experience as they present themselves to consciousness, without recourse to theory, deduction, or assumptions from other disciplines

EthnographyFocuses on the sociology of meaning through close field observation of sociocultural phenomena. Typically, the ethnographer focuses on a community.

Historical

Systematic collection and objective evaluation of data related to past occurrences in order to test hypotheses concerning causes, effects, or trends of these events that may help to explain present events and anticipate future events.

Main Types of Data Collection & Analysis

Those who are not familiar with

qualitative methodology may be

surprised by the sheer volume

of data and the detailed level of

analysis that results even when

research is confined to a small

number of subjects

(Myers, 2002).

“ “

Three Main Methods Of Data Collection

Analysis begins when the data is first collected and is used to guide decisions related to further data collection.

Interactive interviewingPeople asked to verbally described their experiences of phenomenon

Written descriptions by participants

People asked to write descriptions of  their experiences of phenomenon

ObservationDescriptive observations of verbal and non-verbal behavior

In communicating--or generating--the

data, the researcher must make the

process of the study accessible and

write descriptively so tacit knowledge

may best be communicated through the

use of rich, thick descriptions

(Myers, 2002)

“ “

What is Content Analysis?

A formal methodology• To discover, uncover, or answer• Uncover unknown qualities about the data• Looking for patterns in the content

A Little History

Systematic analysis of texts performed several times by religious entities prior to 1900

Major growth periods in the 20th CenturyEarly 20th Century

Studies of newspaper content

Behavior sciences in 1930s and 1940s Begin to study media effects

World War II Propaganda studies

Post war expansion:conversation analysis, personal document analysis, processes of communication, and a generalized measure of meaning

Source: Krippendorf, 2004

More History

1960s: Computer text analysis but challenging beyond quantitative analysis of text

Today:extensive proliferation of traditional, electronic, and social media are leading to strong interest in content analysis and more powerful software

Content Analysis Software Evolution Cards - counting N-vivo – pulled out paragraphs SPSS Text Analytics – computer generated counting

increased the quantity accessible Leximancer – text analytics plus context, same word with

multiple meanings, concepts instead of keywords, interactive and visual

What is Watson?

We have a proliferation of Big Data,”

90% of the world‘s information was created in the last two years

80% of that 90% is unstructured or semi-structured information, like doctor’s notes or product reviews on Amazon

IBM’s Watson is a genius at reading unstructured information

The ability to format and use these knowledge banks will significantly alter how and how well humans make decision.

Application Areas Today

Conversations Conversations

SentimentSentiment

Social MediaSocial Media

ForensicsForensics

Historical ReviewsHistorical Reviews

Political DocumentsPolitical Documents

News AnalysisNews Analysis

PropagandaPropaganda

Television ContentTelevision Content

Bias DeterminationBias Determination

Song LyricsSong Lyrics

National SecurityNational Security

Video Game ContentVideo Game Content

More ...More ...

Content analysis is a formal

methodology to study a collection of

media to discover, uncover, or answer

A Formal Methodology

A formal, objective method with rigor and repeatabilityMany methods and processes are validMethodology example1. Determine research question2. Identify and collect samples3. Perform quantitative analysis4. Perform qualitative analysis5. Draw conclusions6. Summarize, publish, and share results

To Study A Collection of MediaMedia is a method of information communicationCollections include the following (normalize formats to text)

Written media such as newspapers, magazines, websites, Blogs, Tweets, Facebook pages, emails

Audio, such as radio programs, interview transcripts, conversations (can be transcribed into text)

Video, such as television, movies, news footage, YouTube videos (can be transcribed into text)

Images described in text

To Discover, Uncover, Answer...

Discover concepts, themes, and relationships in the collectionUncover unknown qualities about the dataAnswer a specific research question

ConceptsThemes

Relationships

Key Points

All methods of content analysis share common components:

Quantitative (counting) and qualitative (meaning) analyses Analysts can use one or both methods Content analysis is best when both quantitative and

qualitative approaches are combined Important study aspects include

Sampling Units Of Measure Coding Validity Reliability

Sampling: Taking a Subset

Sampling plans are needed to reduce researcher bias

Select a type of sampling (e.g., random)Sample size is important to be representativeSplit-half technique: Two samples equal the same result

Source: Krippendorf, 2004

Units of Measure

Sampling Part 2: Samples require a definition of data resolution

Television comedies, 1/2 hour, Wednesday nightsEntire tweet, tweets from a user, collection of topical tweetsOne blog entry, an entire blog, or consolidation of many blogsNewspaper article, articles of a set timeline

Content analysts must determine these units to measure

Impacts relationships of words and codingConcept discovery restricted to within units

CodingProcess of examining text in a specific unit and extracting relevant dataLook for words, phrases, word sense, and categorize units of text (i.e., words, sentences, paragraphs, tweets)Three methods of coding1.Manual, by person(s) coding from codebooks,

instructional guides, intuition

2.Computer-assisted (N*Vivo) beginning with coding then often some automation for remaining documents

3.Computer generated (Leximancer, CATPAC)

Reliability and ValidityFor a formal analysis method to be sound, reliability and validity must be addressedReliability refers to stability and reproducibility

Coding to be repeatable if manual or computer assistedInter-rater reliability for manual coding with multiple coders affects reproducibility and must be ensuredMeasure of accuracy is tied to statistical normsAccuracy is the strongest form of reliability (Weber, 1990)

Validity

Validity refers to general applicability of results and conclusions obtained from inferences in the studyMajor concern for qualitative analysis in general

Researcher chooses coding concepts --makes inferencesResearcher bias, errors, conclusions

Neuendorf listed external validity, face validity, criterion validity, content validity, and construct validity

Are we measuring what we want to measure? Neuendorf, 2002, p. 112“

“

Quantitative Analysis in Qualitative Research

Counting and statistics: Numeric measures

Word frequenciesHow many times does a word appear?

Specify stop-words to ignore (e.g., the, and, others)

Need to consolidate synonyms, stems (e.g., dog = dogs)

Compound words (i.e., word pairs) are important

United States

not good

Categories (simply present or frequencies)

Quantitative Analysis of Qualitative Information

Concept frequenciesHow often do concepts occur?

Existence (occurs) or actual counts

Other StatisticsProximity and co-occurrence frequencies can all be used to determine concept relationships

Qualitative AnalysisCoding is performed to reduce text collection to categories (i.e., concepts)Analyst can seed concepts or discover concepts during analysisOften, the more discovery allowed the more objective the analysis (grounded theory reduces researcher bias)Concepts and their relationships form the foundations for extracting meaningKeyword in context (KWIC)

Which words and how used (Weber, 1990)

Qualitative AnalysisCoding is performed to reduce text collection to categories (i.e., concepts)Analyst can seed concepts or discover concepts during analysisOften, the more discovery allowed the more objective the analysis (grounded theory reduces researcher bias)Concepts and their relationships form the foundations for extracting meaningKeyword in context (KWIC)

Which words and how used (Weber, 1990)

What is a Concept?Synthesis of a text representation

Key words, including consolidating synonyms, stems

Represents something meaningful

Found by examining word, compound word, and surrounding words in a measurable unit

Useful to display on a graphical “map”

Role of the Computer SolutionsA content analysis can be done without a computer. Although...

At a minimum, a computer serves as a document file folder and backup device

And a search tool for and within documents

Software can also assist with manual coding then continue coding automatically (N*Vivo)

Or software can do coding automated by statistical processing (Leximancer) or networks (CATPAC)

Key Points SummaryA content analysis is best when both quantitative and qualitative approaches are combined (Weber, 1990).

Quantitative analysis counts and finds statistics

Qualitative analysis determines meaning

Important operational aspects include sampling, units of measure, coding, validity, and reliability

References

Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage. Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.Weber, R. P. (1990). Basic content analysis. Newbury Park, CA: Sage.Willig, C. (2008). Introducing qualitative research in psychology: Adventures in theory and method (2nd ed.). Philadelphia, PA: Open University Press.

Additional Reading

Evaluation of Unsupervised Semantic Mapping of Natural Language with Leximancer Concept Mapping, Andrew Smith. Conversations Between Careers and People With Schizophrenia: A Qualitative Analysis Using Leximancer, Julia Cretchley, Cindy Gallois, Helen Chenery, and Andrew Smith Analysis of Asynchronous Discourse in Web-assisted and Web-based Courses, David Thomas and Cleborne MadduxComputer Aided Phenomenography: The Role of Leximancer Computer Software in Phenomenographic Investigation, Sorrel Penn-EdwardsContent Analysis of a Random Day of Two News Sites: FoxNews.com and MSNBC.com, Michael R. Neal

Qualitative Analysis1.Grounded Theory 2.Phenomenological Analysis3.Discourse Analysis4.Narrative Research5. Intuitive Inquiry

THANK YOUTHE PSYCHOLOGY OF AUDIENCE ENGAGEMENT

Dr. Pamela Rutledge | prutledge@fielding.eduDr. Jerri Lynn Hogg | jhogg@fielding.edu