Research Designs Using Content Analysis
Klaus Weber Northwestern University
klausweber@[Link]
PDW Content Analysis in Organizational Research Academy of Management Montreal, 2010
A Definition
Content analysis is a research technique for making replicable and valid inferences from texts [broadly conceived] to the contexts of their use.
Krippendorff, 2004
Technique: Systematic (reliable, valid)
Evidence based, transparency of procedure
Text: Data is generated in communication process
Author, audience, intent, context matter
Inference: Interpretative in nature, inference about social reality
Researcher, research question, theoretical constructs
Examples of how data is generated: Conversations, speeches, articles, open ended survey responses, interviews, images
2
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
Outline
Whats in a Text? A Semiotic Framework Generic Types of Textual Analysis The Process From Collecting Data to Presenting Results Sampling, Base Rates and Control Groups Validity and Reliability Software Support
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
Whats in a Text? A Semiotic Perspective
Conceptual image of innovation
Sense / Concept
Text = words/images arranged in order, but of interest are often ideas or actions that the words point to The semiotic problem: words, concepts, and referents do not correspond one-to-one (the good news: associations are usually conventionally defined in a particular social context) Referent
Innovation activity in organization
Sign Unit Sign vehicle
Word in a Text e.g., innovation
-> Be clear about what the text analysis is getting at: Linguistic patterns, cognitive-cultural schemas, facts?
Solutions to the semiotic problem:
Three generic sources of meaning and interpretation:
Referential: A words meaning derives from its association with a referent or idea (e.g., categories, names) Relational: Meaning derives from a signs position to other signs (e.g., association, opposition, grammar, plots) Contextual: Meaning derives from the communication context (who created the text, for whom, when, why)
Klaus Weber, Content Analysis, Academy of Management Meeting 2010 4
What Content Is Analyzed
Inductive Theory Building
Holistic interpretation (-> themes, mechanisms, taxonomies)
Quantitative Content Analysis
Small text units in isolation, e.g. categories (-> frequencies, trends, etc.)
Semantic Analysis
Relationship between content units, e.g. associations and grammar (-> scripts, networks of associated concepts, causal maps, narratives)
Computational Linguistics
Algorithm based identification of structure, e.g. latent semantic clusters (-> analogue: factor and cluster analysis; problem: interpretation)
Discourse Analysis
Several texts, e.g. regimes of interpretation (-> broad ideologies, institutional myths and political contradictions)
Klaus Weber, Content Analysis, Academy of Management Meeting 2010 5
The General Process
Step
Data Generation & Collection
Activities / Decision Points
Sampling, interviews, archived documents, web content, recordings of conversations & speeches, etc. Digitalization (transcribing, OCR scans); Cleaning and Formatting, Unitization and Indexing (author, document, sentence, etc.)
Data Storage & Organization
Categorizing and Connecting Coding and Aggregation Simplification and Presentation
Category development (custom, standard); Dictionary development (custom, standard); Operational Definition of Associations
Sampling, Coding, Reliability/Validity, Aggregation of unit-level coding
Frequencies, Trends, Comparisons, Networks, Maps, MDS, Statistics, etc.
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
Sampling, Base Rates and Control Groups
Sampling authors
Most common
Sampling texts
No need to code every document (in most instances)
Sampling coding units
No need to code every paragraph / sentence (in most instances)
Base rates of word or category occurrence
Random is fine but often not most realistic
Control groups
Corpus linguistics Closely matched other texts
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
Validity and Reliability of Inference
Validity: external (generalization), internal (constructs and causality) Reliability: replication over time, across individuals
Multiple researchers for category development
Applicability of generic categories and dictionaries
Documentation (code book, dictionary) Reliability statistics: Dont use percent agreement, avoid Cohens kappa; use statistics that account for chance agreement between coders: Krippendorffs alpha, Scotts pi
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
Software Support Options: Some Packages
Functionality Functionality Functionality Functionality Functionality
Type of Software
Theory Building Support Coding Support
Examples of popular software
[Link], Ethonograph, Kwalitan, MaxQDA, NVivo Diction, TextQuest, LIWC Yoshicoder WordStat AutoMap, DecisionExplorer, UCINet
Storage, retrieval
Yes
(best for smaller volumes)
Developing and linking categories
Yes
(main focus)
Automated content coding
Some
(best for smaller volumes)
Mapping, display of coded data
Some
(mostly basic)
Quantification, statistics
Little
(export to other software)
Yes
Little
Yes
(main focus, efficient for high volume)
Little
Little
(export of other software)
Mapping
Some
Little
Yes
Yes
(main focus)
Some
(e.g. concept centralities)
Text Mining
TextAnalyst, SAS plug-in, SPSS plug-in WordStat, TAKMI
Yes
(especially for large volumes)
Little
Yes
Some
Some
(e.g. built in algorithms)
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
A Few Key Questions To Ask
What does the content represent? How were the data generated? Are categories and measurements valid and replicable?
What is the base rate of content occurrence?
What inferences / interpretations can I actually make?
Klaus Weber, Content Analysis, Academy of Management Meeting 2010
10
