Covering the Great Recession was conducted in three parts. The first was made up of coding from PEJ’s weekly News Coverage Index (NCI). That sample analyzed data from February 1 through August 31, 2009 and is referred to here as the broad sample. It consists of all news stories including those that were about subjects unrelated to the economy.
The complete methodology of the News Coverage Index is available here.
The second part of the study includes a closer examination of a sub-section of economic-related stories as they were originally coded in the NCI. This sample covers the time period of February 1 through July 3, 2009 and is referred to here as the economy-focused sample. Details of that process are below.
The third element, an examination of the precise phrases and ideas that resonated most fully in the media was conducted in collaboration with researchers at Cornell University and Stanford University. This third element included an even larger universe of media – more than a million different web-based outlets each day, including RSS feeds from mainstream news sites, new media sites and blogs. The analysis conducted here covers the time period of February 1 through July 3, 2009. More details on their methodology are found in the text box within the main narrative and also on their website, http://memetracker.org.
Sample Design
Broad Sample
The broad sample included all the outlets that are part of PEJ’s regular News Coverage Index.
These outlets, along with the methods of rotation, are as follows:
Newspapers (Sun-Fri)
Coded every day
NY Times
Coded two out of these four every weekday and Sunday
Washington Post
USA Today
Los Angeles Times
Wall Street Journal
Coded two out of these four every weekday and Sunday
Kansas City Star
Pittsburgh Post-Gazette
San Antonio Express-News
San Jose Mercury News
Coded 2 out of these 4 every weekday and Sunday
Herald News (MA)
Anniston Star (AL)
Spokesman-Review (WA)
Meadville Tribune (PA)
Web sites (Coded 6 of 12 each day, Mon-Fri)
CNN.com
Yahoo News
MSNBC.com
Google News
AOL News
Foxnews.com
USAToday.com
Washingtonpost.com
ABCNews.com
BBC News (international version)
Reuters.com
NYTimes.com
Morning Network TV (Mon-Fri)
ABC – Good Morning America
CBS – Early Show
NBC – Today
Evening Network TV (Mon-Fri)
ABC – World News Tonight
CBS – CBS Evening News
NBC – NBC Nightly News
PBS – NewsHour with Jim Lehrer (rotated daily between the first 30 minutes and the second 30 minutes of the hour-long broadcast)
Cable TV (Fifteen in all, Mon-Fri)
Daytime (2:00 to 2:30 pm) coded 2 out of 3 every day
CNN
Fox News
MSNBC
Nighttime CNN – coded 2 out of the 4 every day
Situation Room (6 pm)
Lou Dobbs Tonight
CNN Prime Time/Campbell Brown: No Bias, No Bull
Anderson Cooper 360
Nighttime Fox News – coded 2 out of the 4 every day
Special Report w/ Bret Baier
Fox Report w/ Shepard Smith
O’Reilly Factor
Hannity
Nighttime MSNBC – coded 2 out of the 4 every day
The Ed Show/1600 Pennsylvania Ave
Hardball (7 pm)
Countdown w/ Keith Olbermann
Rachel Maddow
News Radio (Mon-Fri)
NPR Morning Edition every day (rotated daily between the first 30 minutes of the first hour and first 30 minutes of the second hour)
ABC Radio headlines at 9am and 5pm
CBS Radio headlines at 9am and 5pm
Talk Radio (Mon-Fri)
Rush Limbaugh every other day
1 out of 2 additional conservatives each day
Sean Hannity
Michael Savage
1 out of 2 liberals each day
Ed Schultz
Randi Rhodes (Stephanie Miller show was coded from March 2 to May 8 while Randi Rhodes was off the air.)
Story Inclusion
For the broad sample, PEJ analyzed all stories with a national or international focus that appeared as follows:
- On the front page of newspapers
- During the first 30 minutes of network morning news, cable programs, and talk radio shows
- During a thirty minute segment of NPR’s Morning Edition and PBS’ NewsHour with Jim Lehrer
- As one of the top 5 stories on each Web site at the time of capture
- During the entirety of the commercial network evening newscasts and syndicated news headlines segments on ABC radio and CBS radio
Economy-focused Sample
The economy-focused sample was made up of stories coded in the broad sample from February 1 through July 3, 2009 that fell into a list of economy-related storylines. Those included:
Economic crisis
U.S. auto industry
Madoff banking scam
Gas/oil prices
Energy
Taxes/Tax Day
U.S. airline industry
Fannie Mae/Freddie Mac troubles
Effect of the economic crisis on health care
Effect of the economic crisis on education
Economy-related stories about the April G-20 summit
To arrive at the economy-focused sample, stories from talk radio and all broadcast stories that were 20 seconds or less were excluded from the sample. Then, every 4th story was selected randomly from each of the outlets.
The resulting economy-focused sample consisted of 1,656 stories.
Capture and Retrieval
All outlets are captured and included in PEJ’s media archive.
For newspapers that are available in print in the Washington, D.C. area, hard copies are used. For newspapers that are not available for delivery, digital editions of the paper are retrieved either through the newspaper’s own Web site, or through the use of digital delivery services such as pressdisplay.com and newsstand.com. When necessary, the text of articles are supplemented by the archives available in the LexisNexis computer database.
Radio programs are captured through online streams of the shows. Using automated software, we record several local affiliates that air the program in various markets throughout the country. The purpose of this method is to ensure that we have a version of the program in case one of the streams is unavailable on a particular day, and so that we record the show in a manner that represents the way a typical listener would hear the program with commercials and newsbreaks.
Online websites are captured manually by a member of PEJ’s staff. The capture time is rotated daily between 9-10 am ET and 4-5 pm ET. The home pages and pages with the top articles for all sites are saved so that when we reference the material, the format is the same as it appeared online at the time of capture.
Finally, all television shows are recorded digitally and archived for coding purposes. PEJ is a subscriber to DirectTV satellite service and all programs are recorded onto multiple TiVo recording units before being burned onto DVDs for archival purposes.
All television and radio programs are then coded by a member of PEJ’s staff who watches or listens to the archived version of the program.
Coding Team & Process for Weekly Index Coding
The data derived from PEJ’s regular Index coding was conducted by PEJ’s team of 15 trained coders. We have tested all of the variables contained in the regular weekly Index coding and all the variables reached a level of agreement of 80% or higher. For specific information about those tests, see the methodology section for the NCI.
Additional Coding of Economy-focused stories
For the stories from the economy-focused sample (1,656 stories), additional coding was conducted for three additional variables.
Stories were coded for dateline, trigger, and sources.
- Dateline refers to the geographical location of where the report was conducted.
- Trigger designates the action, event, or editorial decision that makes this news, thus triggering its publication.
- Sources refer to whether a particular person or group was cited as a direct or indirect source within the story. Multiple categories of sources could appear in the same story.
Coding Team & Process for the Additional Coding
A team of five of PEJ’s experienced coders worked with a coding administrator in order to complete the additional coding for this particular study.
In addition to the main intercoder testing conducted on all NCI variables, supplemental testing was conducted on the additional variables used in this portion of the study. 30 randomly selected stories were coded by all members of the coding team.
The percent agreement for each variable was as follows:
Dateline: 95%
Trigger: 82%
Sources (all combined): 96%
Individual source categories
Obama: 97%
White House West Wing Other: 99%
Congress/Democrat: 100%
Congress/Republican: 100%
Congress/combined: 100%
Federal agency: 93%
State/local government: 90%
Democratic party/unelected: 97%
Republican party/unelected: 98%
3rd party: 100%
Business: 86%
Organized Labor/Union: 100%
Academic/Independent Expert: 84%
Citizens/workers: 95%
Interest groups: 94%
Other media: 96%
Unnamed sources only: 98%
