The Master Character Narratives in Campaign 2012

About This Study

A number of people at the Pew Research Center's Project for Excellence in Journalism worked on PEJ's "The Master Character Narratives in Campaign 2012." Director Tom Rosenstiel and Associate Director Mark Jurkowitz wrote the report. Senior Researcher Paul Hitlin supervised the content analysis component. Researchers Steve Adams, Heather Brown, Emily Guskin and Sovini Tan coded and analyzed the content data. Monica Anderson created the charts. Jesse Holcomb copy edited the report. Dana Page handles the communications for the project.

Methodology

The report issued by the Pew Research Center's Project for Excellence in Journalism, "The Master Character Narratives in Campaign 2012," uses content analysis data derived from human coding.

Sample Design

The content was based on media coverage originally captured as part of PEJ's weekly News Coverage Index (NCI) from May 29-August 5, 2012. 

Each week, the NCI examines the coverage from 52 outlets in five media sectors, including newspapers, online news, network TV, cable TV, and radio. Following a system of rotation, between 25 and 28 outlets each weekday are studied as well as 3 newspapers each Sunday.

For this particular study of campaign coverage, ABC and CBS radio headlines were excluded. Therefore, the 50 media outlets examined for this campaign study were as follows:

Newspapers (Eleven in all)

Coded two out of these four every weekday; one on Sunday
The New York Times
Los Angeles Times
USA Today
The Wall Street Journal

Coded two out of these four every weekday; one on Sunday
The Washington Post
The Denver Post
Houston Chronicle
Orlando Sentinel

Coded one out of these three every weekday and Sunday
Traverse City Record-Eagle (MI)
The Daily Herald (WA)
The Eagle-Tribune (MA)

Web sites (Coded six of twelve each weekday)

Yahoo News
MSNBC.com
CNN.com
NYTimes.com
Google News
FoxNews.com

ABCNews.com
USAToday.com
WashingtonPost.com
LATimes.com
HuffingtonPost.com
Wall Street Journal Online

Network TV (Seven in all, Mon-Fri)

Morning shows - coded one or two every weekday
ABC - Good Morning America
CBS - Early Show
NBC - Today

Evening news - coded two of three every weekday
ABC - World News Tonight
CBS - CBS Evening News
NBC - NBC Nightly News

Coded two consecutive days, then skip one
PBS - NewsHour

Cable TV (Fifteen in all, Mon-Fri)

Daytime (2:00 to 2:30 pm) coded two out of three every weekday
CNN
Fox News
MSNBC

Nighttime CNN - coded one or two out of the four every day

Situation Room (5 pm)
Situation Room (6 pm)
Erin Burnett OutFront
Anderson Cooper 360

Nighttime Fox News - coded two out of the four every day
Special Report w/ Bret Baier
Fox Report w/ Shepard Smith
O'Reilly Factor
Hannity

Nighttime MSNBC - coded one or two out of the four every day
PoliticsNation
Hardball (7 pm)
The Rachel Maddow Show
The Ed Show

Radio (Seven in all, Mon-Fri)

NPR - Coded one of the two every weekday

Morning Edition

All Things Considered

Talk Radio
Rotate between:

Rush Limbaugh
Sean Hannity

Coded every other day
Ed Schultz

From that content, the study included all campaign-related stories:

• On the front page of newspapers
• In the entirety of commercial network evening newscasts
• The first 30 minutes of network morning news and all cable programs, along with the PBS evening news
• The first 30 minutes of the talk radio programs
• A 30 minute segment of NPR's broadcasts or PBS' NewsHour (rotated between the first and second half of the programs)
• The top 5 stories on each website at the time of capture

Click here for the full methodology regarding the News Coverage Index and the justification for the choices of outlets studied.

Sample Selection

To arrive at the sample for this particular study of campaign coverage, we began by pulling all the stories from May 29-August 5, 2012, that were either coded as campaign stories, meaning that 50% or more of the story was devoted to discussion of the ongoing presidential campaign, or included President Obama or Governor Romney in at least 25% of the story.

For all stories, further sampling was conducted by selecting every other relevant story by outlet. This was done by listing the stories from each show in chronological order and randomly selecting the first story. We then selected every-other story within each outlet to arrive at the final sample.

This process resulted in a sample of 827 stories.

Coding Design

A coding protocol was designed for this project based on previous studies by PEJ.

For each of the two major party presidential candidates (Barack Obama, Mitt Romney), researchers examined campaign coverage to identify the ten most common themes about the character and record of each candidate. Five of the themes for each candidate were positive and five negative. (For Obama, two themes about his economic policy were eventually merged.) Researchers then identified each assertion within every story reflecting any of these themes in the sample.

Unit of Analysis
The unit of analysis for this study was the assertion (reflecting one of the major narrative themes). Any time a coder came across an assertion within one of the stories in the sample, he or she noted the assertion and coded the variables listed below for each one. Many stories contained multiple assertions about a candidate; many other stories did not have any assertions about character and record relevant for this study.

Variables
The variables included in this study were the following: theme/message, affirming the narrative thread or rebuttal, and variable source of statement.

Theme/message are the overarching themes that the press, candidates, and operatives focus on and are usually centered on a candidate's character. As noted above, the master themes about each of the two candidates were identified and tested prior to coding. This variable measures which of the themes the statement addressed.

Affirming the narrative thread or rebuttal captures whether the assertion being coded is affirming the narrative thread or refuting it.

Source of statement designates the person who is making the statement. This is not necessarily the author of the piece, but the person who expresses the particular statement. For example, if a newspaper story quotes an unnamed voter as asserting that, "Senator Obama cares about regular people," that means the source of the statement is the voter and not the journalist writing the piece.

Coding Team & Process

The team responsible for performing the content analysis on this particular study was made up of four experienced coders and a senior researcher.

Testing of all variables used to determine campaign stories has shown levels of agreement of agreement of 80% or higher. For specific information about those tests, see the methodology on intercoder testing.

During coder training for this particular study, intercoder reliability tests were conducted for all the campaign-specific variables. There were two different intercoder tests conducted to assure reliability.

The first test was to assure that coders could identify the assertions within campaign stories. Each coder was given the same group of stories and asked to identify where threads appeared in those stories. The agreement between all the coders on this task was 80%.

The second test consisted of each coder being given a list of assertions and asked to code each of the campaign specific variables for those threads.

From that test, the specific levels of agreement for the variables in this study were as follows:

Thread/statement: 87%
Affirming narrative thread or rebuttal: 96%
Source of statement: 80%

Additional Analysis

The results of PEJ's coding were twinned with a companion survey of public attitudes about the candidates by the Pew Research Center for the People & the Press. The survey of registered voters was conducted on August 16-19, 2012. Together, the two studies allowed us to explore how much these press messages were shaping public opinion of the candidates.