Content Analysis
Sample Design
A content analysis of coverage of the 2004 presidential campaign was conducted via 31 media outlets-six newspapers, 20 television and radio programs (19 television and 1 radio), and five Internet sites.
For newspapers, the New York Times and the Washington Post were selected based on their national reputation for political coverage. Four other newspapers were selected from a random sample of large regional newspapers. The sample ensures geographic and ownership diversity. These newspapers are also included in the Project's Annual Report on the State of the News Media. The six papers studied were as follows:
Television and radio programs, selected to provide the widest range of coverage, included the following:
We also examined five Internet blogs
The outlets were monitored for four weeks of coverage between March and June 2004. The first period was from March 25 to March 31. The second period was from April 8 to April 14, the third period was from May 13 through May 19. The fourth week was from June 1 to June 5 (June 6th and 7th of this week were omitted because President Reagan's death on the 5th halted campaign activities for several days.).
Inclusion and Screening
Newspaper articles for the selected days were downloaded from Lexis-Nexis using the search terms "Bush or Kerry." These articles were then screened to eliminate stories where these terms were used in a non-relevant context (such as stories about the baseball player Kerry Wood or gardening stories mentioning "bushes"), stories where either President Bush or Senator Kerry are mentioned only in passing (such as a reference to a school that had been visited by President Bush), and stories on Bush administration actions with no analysis or discussion of their political dimensions. This initial screening yielded 1,006 newspaper stories.
Transcripts of television and radio news programs were downloaded from Lexis-Nexis and screened using the same procedure. Programs archived in Lexis story-by-story yielded 565 stories. Lexis archives five of the programs, Nightline, Hardball, Aaron Brown, Larry King and the NewsHour as one program-long story, adding another 80 "stories."
Daily transcripts for each of the five blogs were captured from the individual websites. Each day that a blog was updated counted as one story, regardless of the number of postings per day. This totaled 113 initial stories.
Next coders read each story individually to search for mentions of the seven themes in our study: Bush lacks credibility, Bush is arrogant, Bush is a strong and decisive leader, Kerry is very liberal, Kerry is an elitist who is not like other people, Kerry hems and haws, and Kerry is a tough guy who won't back down.
Rather than code the story as a whole, we wanted to examine the treatment of each individual assertion. Therefore, every time a relevant statement emerged, it was highlighted for inclusion. To be included, a statement needed to assert or refute one of the seven character themes. There was a wide range of acceptable language for each theme. For example, the wording of a statement about Bush as arrogant could be that he is flippant, ignores reality, won't admit mistakes, charges ahead, is impudent, insensitive, uncaring, etc.
Some stories had three or four assertions, while others had just one. Stories with no assertions were discarded. This resulted in a project sample of 506 stories and 1,073 total assertions, 955 affirming and 118 rebuttals. The theme-by-theme breakdown is as follows:
Occurrences of this last theme, Kerry is an elitist, were too few for individual analysis.
Coding Process
Each of the 1,073 assertions was coded individually be a team of researchers at the Center for Advanced Social Research of the University of Missouri School of Journalism. The project managers were Ken Fleming, director of the Center and Esther Thorson, associate dean of graduate studies and research at the school.
A description of the study's objectives and data processing was provided to orient the five coders and project managers of the content analysis project. The codebook contained a dictionary of coding variables, values, and operational definitions with detailed instructions to the coders. Prior to the actual coding process, extensive training was conducted with the coders. Meetings were held at least once a week between the coders and project managers to discuss problems and issues identified during the coding process. Coding sheets were designed and used throughout the four-week period of time. Reliability check was conducted every week using a random selection of sample stories for each of the twelve coding variables. The overall intercoder reliability of the content analysis was .851, calculated with Scott's Pi.
Late Night Show Analysis
The opening monologues of the three late-night comedy shows-The Tonight Show, The Late Show, and The Daily Show were studied for the presence of the seven narrative themes. Video tapes of the shows were used for the analysis. Program re-runs aired during the content analysis period were discarded. In all, 33 shows were studied, broken down as follows: The Daily Show-9, Late Night with David Letterman-11, The Tonight Show with Jay Leno-13.
Ad Campaign Analysis
Bush and Kerry campaign ads were also analyzed for references to the seven character themes. All campaign ads were captured via the candidates' own websites. All ads listed as airing from February 14 through June 5th were studied.
Public Opinion Survey
The public opinion component of this report was conducted by the Pew Research Center for the People and the Press as a part of the June 2004 Voter Attitudes Survey. The survey was conducted from June 3-13, 2004. The total for questions about the six themes was 1330. Esther Thorson of Missouri performed further statistical analysis of the survey data. She looked at how demographics, party affiliation, political ad exposure (Kerry and Bush ads), and attention to news were related to both individual responses to each narrative theme (for example, when you hear a candidate flip flops, would you say that was Kerry or Bush or neither or both?), and how they were related to the total number of narrative themes that each person "correctly" connected the candidates. Two different types of regression were performed:
Overall Analysis
To test what features were associated with people identifying each theme characteristic with the candidate it had been linked with (i.e., Kerry with flip flopping; Bush with refusing to admit he was wrong), we added up for each respondent how many characteristic-candidate links they got correct (which ranged from 0 to 6). We then used hierarchical linear regression to see first how demographics affected the scores, then how party affiliation affected them, and finally how much Kerry and Bush advertising people had seen and how much attention they reported paying attention to the news.
In the regression, the dependent variable, the number of themes each individual linked with the candidate defined in the theme, ranged from 0 to 6.
In a hierarchical multiple regression, the first block of variables entered included age, gender, (defined as 1 = male; 0 = female), education (1 = Below high school; 2 = High school graduate, 3 = Some college, 4 = College degree, and 5 = Post college), and race (1 = white; 0 = non-white).
In the second block of variables, we entered which candidate the respondent said they planned to vote for (Bush = 1; undecided = 0; and Kerry = -1).
In the third block we entered how many Kerry ads they had seen (3 = A lot, 2 = Some, and 1 Only once or twice), how many Bush ads they had seen (3 = A lot, 2 = Some, 1 = Only once or twice), and how much attention they had paid to four news areas: News about Presidential candidates; News about Iraq; News about the high price of gasoline; and News about the abuse of Iraqi prisoners by U.S. soldiers in Iraq. The scale for each news area was 3 = Follow the news story very closely, 2 = Follow fairly closely, 1= Follow not too closely, and 0 = Not at all closely. Thus the scale values varied from 0 to 12.
Theme by Theme Analysis
To test what features were associated with people identifying each theme as characterizing Bush or Kerry or neither or both, the following analysis was undertaken. First, we eliminated all respondents who answered neither or both because for them, there really was no "narrative theme" about either of the candidates.
Next we coded the dependent variable for each theme as choosing Kerry = 1, and choosing Bush = 0. Then we applied the same hierarchical multiple regression model used in the "corrects" analysis. That is,
The first block of variables entered included age, gender, (defined as 1 = male; 0 = female), education (1 = Below high school; 2 = High school graduate, 3 = Some college, 4 = College degree, and 5 = Post college), and race (1 = white; 0 = non-white).
In the second block of variables, we entered which candidate the respondent said they planned to vote for (Bush = 1; undecided = 0; and Kerry = -1).
In the third block we entered how many Kerry ads they had seen (3 = A lot, 2 = Some, and 1 Only once or twice), how many Bush ads they had seen (3 = A lot, 2 = Some, 1 = Only once or twice), and how much attention they had paid to four news areas: News about Presidential candidates; News about Iraq; News about the high price of gasoline; and News about the abuse of Iraqi prisoners by U.S. soldiers in Iraq. The scale for each news area was 3 = Follow the news story very closely, 2 = Follow fairly closely, 1= Follow not too closely, and 0 = Not at all closely. Thus the scale values varied from 0 to 12.
For the Pew Survey, battleground states included the following: AZ, AR, FL, IA, LA, ME, MI, MN, MO, NV, NM, NH, OH, OR, PA, TN, WA, WV, WI
The full survey and methodology can be found at http://people-press.org/reports/methodology.php3 [1]