content analysis: measurement and validity
DESCRIPTION
Content Analysis: Measurement and Validity. Kimberly A. Neuendorf, Ph.D. Cleveland State University Fall 2011. Measurement. “Assignment of numerals to objects or events according to rules” (Stevens, 1951). Operationalization (Measurement). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/1.jpg)
Content Analysis:Measurement and
ValidityKimberly A. Neuendorf, Ph.D.
Cleveland State UniversityFall 2011
![Page 2: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/2.jpg)
Measurement “Assignment of numerals to objects or
events according to rules” (Stevens, 1951)
![Page 3: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/3.jpg)
Operationalization (Measurement)
Categories that are exhaustive and mutually exclusive
High and appropriate level of measurement Ratio Interval Ordinal Nominal
![Page 4: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/4.jpg)
Evaluating Measures:Comparing Reliability, Accuracy (lack of Bias), and Precision
Targets…also see p. 114 of book. See also:
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park, CA: Sage Publications.
![Page 5: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/5.jpg)
![Page 6: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/6.jpg)
High Reliability
High Accuracy
High Precision
![Page 7: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/7.jpg)
![Page 8: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/8.jpg)
High Reliability
Low Accuracy
High Precision
![Page 9: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/9.jpg)
![Page 10: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/10.jpg)
Low Reliability
Moderate Accuracy
High Precision
![Page 11: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/11.jpg)
![Page 12: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/12.jpg)
Reliability?
High Accuracy
Low Precision
![Page 13: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/13.jpg)
Validity
Internal Validity Conceptual definition =matches= Operationalization The standard of having a “good” measure—”Are we hitting
the target we want?”
External Validity Generalizability
Sample Ecological validity Replicability
![Page 14: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/14.jpg)
Validity
“Face” Validity Criterion Validity
Relate to established standard or behavior Concurrent Predictive
e.g., Oliver & Kalyanaraman (2002) study of sexual and violent content in trailers and films. . . Criterion is the MPAA rating
![Page 15: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/15.jpg)
Validity
Content Validity
Construct = ???
![Page 16: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/16.jpg)
Validity
Content Validity
![Page 17: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/17.jpg)
Validity
Content Validity
![Page 18: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/18.jpg)
Validity
Content Validity
High internal consistency reliability. . .
![Page 19: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/19.jpg)
Validity
Construct Validity Related to other measures (constructs) in ways
consistent with theory Convergent—relationships as expected Discriminant—lack of relationships, as expected
![Page 20: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/20.jpg)
Construct Validity Smith volume (1992) on “thematic content
analysis”—psychographic applications—has examples Trust-mistrust coding system for TAT stories
construct validated against: Other thematic coding systems (e.g., Affiliation; Intimacy) Demographics (e.g., Mothers’ feeding habits during infancy
(more or less principled), moderated by gender & age) Self-reported behaviors (e.g., Reported positive relationships
with others; Divorce, family fights in last year; Drug use) Personality measures (e.g., Impulsivity; Locus of Control)
![Page 21: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/21.jpg)
Validity Assessment & CATA
Short et al. (2010) provides a model for validity and CATA, using tests of a new dictionary for “entrepreneurial orientation” Content Validity
Deductive Inductive
External Validity Reliability Dimensionality (a type of Construct Validity--Discriminant) Predictive Validity (seems to really be Construct Validity—
Convergent???)
![Page 22: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/22.jpg)
Validity Assessment & CATA
A Slight Adaptation of Short et al. (2010), converging with Carmines and Zeller (1979): Face Validity Content Validity
Deductive Inductive
External Validity Construct Validity--Dimensionality (Discriminant Validity) Construct Validity--Predictive (Convergent Validity) Criterion Validity (Concurrent or Predictive) [Reliability—separate, but without it, validity is also
compromised]
![Page 23: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/23.jpg)
Apply to the iCarly Case? (Disrespectful Behaviors) A Slight Adaptation of Short et al. (2010), converging with
Carmines and Zeller (1979): Face Validity Content Validity
Deductive Inductive
External Validity Construct Validity--Dimensionality (Discriminant Validity) Construct Validity--Predictive (Convergent Validity) Criterion Validity (Concurrent or Predictive) [Reliability—separate, but without it, validity is also
compromised]
![Page 24: Content Analysis: Measurement and Validity](https://reader036.vdocument.in/reader036/viewer/2022081417/56813bb8550346895da4e850/html5/thumbnails/24.jpg)
pause