Campaign for President Takes Center Stage in Coverage Quarterly Report on the News Methodology
As a special report for PEJ’s weekly News Coverage Index (NCI), the Quarterly Report is based on the aggregated data collected from April 1, 2007 – June 29, 2007 (the 2nd quarter of 2007). Examining the news agenda of 48 different outlets in five media sectors, including newspapers, online, network TV, cable TV, and radio, the NCI is designed to provide news consumers, journalists and researchers with hard data about what stories and topics the media are covering, the trajectories of major stories and differences among news platforms. Following a rotation system, 35 outlets each weekday were selected as well as 7 newspapers each Sunday. For its News Index PEJ monitors 48 different news outlets each week (35 per week-day) from five different media sectors Newspapers (Thirteen in all, Sun-Fri) NY Times every day
Code 2 out of these 4 every day
Code 2 out of these 4 every day
Code 2 out of these 4 every day Web sites (Five in all, Mon-Fri) CNN.com Network TV (Seven in all, Mon-Fri) Morning shows
Evening news Cable TV (Fifteen in all, Mon-Fri)
Daytime (1–1:30 pm) – code 2 out of 3 every day
Nighttime CNN – code 3 out of the 4 every day Nighttime Fox News – code 3 out of the 4 every day
Nighttime MSNBC – code 2 out of the 4 every day Radio (Eight in all, Mon-Fri)
Headlines every day
Talk Radio
1 out of 2 additional conservatives each day
1 out of 2 liberals each day From that content, PEJ analyzes all stories with a national or international focus that appearing as follows:
The resulting universe of stories was coded by a team, which is made up of 8 trained coders, a coding administrator, and a senior research methodologist. The complete methodology for the weekly NCI has further details on the coding system and coder reliability.
This report aggregates the NCI from April 1, 2007- June 29, 2007. The resulting universe totals 18,010 news stories, 459 hours of broadcast content (148 hours from network TV, 224 hours from cable, 87 hours from radio), 2.16 million words in newspapers, and 1.10 million words from news websites. The following is some additional methodology information that applies specifically to the quarterly report. Intercoder Reliability Testing In order to continue to assure that our coding is performed with a high level of reliability, we have performed multiple tests of intercoder agreement with all of our coders.
During the 2nd quarter of 2007 we have had 8 professional coders work on the Weekly News Index project. All were included in our intercoder testing. Housekeeping Variables We first conducted a test of the “housekeeping” variables of the index in April. For this test, we selected a random sample of 151 stories from each of the 5 media sectors we cover. Each story was coded by two different coders. This represented more than 10% of the number of stories we code in a given week. Of those stories, 32 were print stories (newspaper and online) and 119 were broadcast stories (television and radio). For our housekeeping variables, we achieved the following levels of agreement:
Print (32 cases)
Broadcast (119 cases)
Main Variables
Having demonstrated that we had a high level of agreement for all of our housekeeping variables, we then had the coders participate in an additional test to determine the level of agreement for the main variables of the index. We randomly selected 116 stories from both print and broadcast mediums, which represent about 8% of the stories we code in a typical week. The level of agreement for each of our key variables was as follows:
We will continue to conduct further tests of intercoder agreement in the coming months in order to assure continued quality in our coding processes. Lexis-Nexis Search for Candidate Mentions Note: The following methodology applies only to PEJ’s search for candidate names (LINK) and not to the majority of data included in this report and the weekly News Coverage Index reports. For a detailed methodology about how the weekly News Index data is compiled, go here. The Lexis-Nexis database search was conducted for each of the three leading candidates, from both the Republican and Democratic fields; and also for Fred Thompson and Mayor Bloomberg. The newspapers included in this search were: New York Times We also searched news transcripts from the three major network television stations. The transcripts for the network television stations include a number of programs that are not included in PEJ’s weekly sample. Also, Lexis has transcripts for the entire programs so some segments that do not appear in the weekly index coding would be included in this Lexis search. The television programs included in the search were: ABC
World News Tonight CBS
60 Minutes NBC
Meet the Press PBS NewsHour with Jim Lehrer For both newspapers and the television transcripts, searches were conducted using the last names of the candidates that appeared in either the headline or lead paragraphs of the story. The reason for searching for the names in the headline or lead paragraphs was to determine the number of stories that focused on the candidates. Another option would have been to search for mentions anywhere in the article or transcript which would have yielded more results, but would have also included many stories where the candidates were not central to the story. The exceptions for using the candidate’s last name were Hillary Clinton; John Edwards Fred Thompson, and Mayor Bloomberg. Because of former President Bill Clinton, a search of the name “Clinton” would yield many articles about him and not the current Senator from New York. In addition, many articles refer to the candidate as “Hillary Clinton” while others refer to her as “Hillary Rodham Clinton”. Therefore, we searched for both options in order to get the proper total number of stories. In locating stories on “John Edwards” and “Fred Thompson”, full names of the candidates were used due in part that Edwards and Thompson are both widely used last names in the US. For Mayor Bloomberg, “Mayor Bloomberg” was used as a search term to avoid including stories on Bloomberg industries or individuals with the same last name. Many articles refer to Michael Bloomberg as “Mayor Bloomberg." 
Campaign for President Takes Center Stage in Coverage |
 Home | Numbers | Analysis | News Index | Daily Briefing | State of the News Media | Journalism Resources | About PEJ | Contact Us
 Powered by: Phase2 Technology
|
