When Technology Makes HeadlinesAbout this Study
A number of members of the PEJ staff assisted in the production of this report, “When Technology Makes Headlines: The Media’s Double-Vision About the Digital Age.” The team leaders on the project were research analyst Kenny Olmstead and Senior Researcher Paul Hitlin. Sovini Tan, Mahvish Khan and Josh Appelbaum aided in the research, including coding and content analysis of 437 stories about technology. Other staff members who made substantial contributions to the report were: weekly News Index manager Tricia Sartor, analyst/coder Laura Houston Santhanam, deputy director Amy Mitchell and director Tom Rosenstiel. When Technology Makes Headlines: The Media’s Double-vision About the Digital Age was conducted in three parts. The first was made up of coding from PEJ’s weekly News Coverage Index (NCI). That sample analyzed data from June 1, 2009 through June 30, 2010 and is referred to here as the broad sample. It consists of all news stories including those that were about subjects unrelated to technology.
The second part of the study includes a closer examination of a sub-section of technology-related stories as they were originally coded in the NCI. This sample covers the time period of June 1, 2009 through June 30, 2010 and is referred to here as the technology-focused sample. Details of that process are below. Details of each of the three analyses follow:
Broad Sample
The broad sample included all the outlets that are part of PEJ’s regular News Coverage Index. The complete methodology of the News Coverage Index is available here. At the beginning of each year, PEJ reexamines the outlets including in the NCI sample and makes changes in order to keep up with circulation and viewership trends in the media. Because this particular study includes data from both 2009 and 2010, the makeup of the sample differs slightly during those two years. In 2009, these outlets, along with the methods of rotation, were as follows:
Newspapers (Sun-Fri)
Coded two out of these four every weekday and Sunday
Coded two out of these four every weekday and Sunday
Coded 2 out of these 4 every weekday and Sunday
Web sites (Coded 6 of 12 each day, Mon-Fri)
Morning Network TV (Mon-Fri)
Evening Network TV (Mon-Fri)
Cable TV (Fifteen in all, Mon-Fri)
Nighttime CNN – coded 2 out of the 4 every day
Nighttime Fox News – coded 2 out of the 4 every day
Nighttime MSNBC – coded 2 out of the 4 every day
News Radio (Mon-Fri)
Talk Radio (Mon-Fri) In 2010, the outlets, along with the methods of rotation, changed to the following:
Newspapers (Sun-Fri) Coded two out of these four every weekday and Sunday Coded 1 or 2 out of these 3 every weekday and Sunday Web sites (Coded 6 of 12 each day, Mon-Fri) Morning Network TV (Mon-Fri) Evening Network TV (Mon-Fri) PBS Cable TV (Fifteen in all, Mon-Fri) Nighttime CNN – coded 2 out of the 4 every day Nighttime MSNBC – coded 2 out of the 4 every day News Radio (Mon-Fri) Radio Headlines Talk Radio (Mon-Fri) 1 liberal every other day
Story Inclusion
Technology-focused Sample
The technology-focused sample constitutes the main analysis for this report. It was made up of stories coded as technology stories in the broad sample from June 1, 2009 through June 30, 2010. First, we gathered the stories that were coded with a broad topic of Technology or Technology and Culture. In addition to these two broad topic areas we also included content identified as technology-related through the more specific variable of “storyline.” The storyline variable tracks specific events in the news. At times, technology-related events fell under different broad topic areas such as business or crime. This two-layer approach ensures that we cast as wide a net as possible in identify the content to analyze. The list of technology-related storylines was as follows:
Cyberspace Issues In all, 953 stories were identified as technology-related. From this group of 953 stories, we next selected every other story to examine in greater detail through additional content analysis. Before coding, though, a trained coder examined each of the selected stories to make sure that it applied to the subject matter in this study. In all, the technology-focused sample included 437 stories.
Capture and Retrieval For newspapers that are available in print in the Washington, D.C. area, hard copies are used. For newspapers that are not available for delivery, digital editions of the paper are retrieved either through the newspaper’s own Web site, or through the use of digital delivery services such as pressdisplay.com and newsstand.com. When necessary, the text of articles are supplemented by the archives available in the LexisNexis computer database. Radio programs are captured through online streams of the shows. Using automated software, we record several local affiliates that air the program in various markets throughout the country. The purpose of this method is to ensure that we have a version of the program in case one of the streams is unavailable on a particular day, and so that we record the show in a manner that represents the way a typical listener would hear the program with commercials and newsbreaks. Online websites are captured manually by a member of PEJ’s staff. The capture time is rotated daily between 9-10 am ET and 4-5 pm ET. The home pages and pages with the top articles for all sites are saved so that when we reference the material, the format is the same as it appeared online at the time of capture. Finally, all television shows are recorded digitally and archived for coding purposes. PEJ is a subscriber to DirectTV satellite service and all programs are recorded onto multiple TiVo recording units before being burned onto DVDs for archival purposes. All television and radio programs are then coded by a member of PEJ’s staff who watches or listens to the archived version of the program.
Coding Team & Process for Weekly Index Coding
Additional Coding of Technology-focused stories • Tech storyline refers to particular storylines that occurred often in news media during the time period under study • Tech area involves the issue or larger subject being covered • Lead company designates the company that is the main focus of the story • Presence of companies identifies whether any of the major tech companies tracked (Apple, Google, Twitter, Facebook, and Microsoft) where present in at least 25% of the story • General technology thread refers to the concepts or impressions that form around technology as a whole in a given story • Company threads refers to concepts or impressions that form around a particular company in a given story
Coding Team & Process for the Additional Coding In addition to the main intercoder testing conducted on all NCI variables, supplemental testing was conducted on the additional variables used in this portion of the study. For the following codes, 30 randomly selected stories were coded by all members of the coding team. The percent agreement for each variable was as follows:
Tech storyline: 91% Specific company threads Google is innovative and its products are superior/better designed: 97%
Social Media Sample
The sections in this report about social media are based off the data collected from PEJ’s New Media Index. The NMI is a weekly report that captures the leading commentary of blogs and social media sites focused on news and compares those subjects to coverage in the mainstream press. This study aggregates the weekly data collected for the News Media Index from June 1, 2009, through June 25, 2010.
Universe Two prominent Web tracking sites, Technorati and Icerocket, monitor millions of blogs and pieces of social media, using the links to articles embedded on these sites as a proxy for determining what these subjects are. The website Tweetmeme uses a similar method to monitor the popular links on the social networking site Twitter. Each of these sites offers lists of the most linked-to news stories based on the number of blogs, tweets, or other pages that link to them. PEJ does not determine what constitutes a “news” story (as opposed to some other topic), but rather relies on the classifications used by each of the tracking sites. A PEJ staff member manually captured the lists from each site every weekday between 9 and 10 am ET. From those lists, the top five linked to articles were captured for further analysis by PEJ staff (SEE BELOW). Through July 3, 2009, PEJ captured information about blogs from both Technorati and Icerocket. However, the relevant component of Technorati’s site stopped working in early July and has been down ever since. Therefore, the NMI reports beginning the week of July 6-10, 2009, only included blog data from Icerocket.
Coding Procedures Almost all of the codes and rules are the same as with the NCI. The variables coded in both projects include story date, source, story word count, story format, story describer, big story and broad story topic. In order to meet high standards of reliability, these variables are all included as part of PEJ’s continuing intercoder testing involving 15 coders and reached levels of agreement above 80%. For more details about PEJ’s intercoder testing procedures for these codes, refer to the detailed methodology about the News Coverage Index. The only additional variables used in the NMI were identifying the original outlet of the news story and tracking the number of links aimed at each story included in the sample. Technorati, Icerocket, and Tweetmeme each provided the number of links within their lists.
Calculations The calculations for the NCI have a different base. That Index measures the time (in seconds) or space (in words) of each story. That is then used to calculate the percent of newshole devoted to each topic. The reason that the New Media Index uses a different measure, links rather than newshole, is because the nature of online media is different from other traditional forms of media. First, there is no limit to the amount of space that can be devoted to a specific story. In a newspaper, there is a limited amount of space on a front page, for example, and a television newscast is limited by its allotted amount of time. Web sites have no such limits. Second, PEJ determined that in this procedure, the number of blogs that link to a news article are a far greater measure of the significance of that article online than the length of the story. A particular article might be quite long in terms of number of words, but if only a few blogs link to it, that article would have only a small influence in the new media environment. A short story that gets linked to many times has a far greater influence. The percent of links for each big story is determined by taking the total number of links in the sample and then dividing that number by the number of links devoted to each specific big story. The percentages are then ranked in order to discover the five storylines that were most present in online commentary.
Differences from the NCI 1. The capture times for the Web sites included in the News Coverage Index rotate each day. In the New Media Index the times are the same each day. Since these lists compile the number of links to stories over a 48-hour window, rotating the time of capture would result in different increments of times between each capture. Through testing, PEJ has discovered that the stories on the lists change significantly more over a 24-hour period than they do over a 12 or 16-hour period. Thus it is more methodologically sound to capture at the same time each day. 2. The News Coverage Index is comprised of primarily U.S.-based media outlets, but the aggregators of blogs and social media include both U.S. and non-U.S. blogs. In addition, stories that are linked-to can be from non-U.S. sources. 3. PEJ’s weekly News Coverage Index includes Sunday newspapers while the New Media Index is Monday through Friday. |
 Home | Numbers | Analysis | News Index | Daily Briefing | State of the News Media | Journalism Resources | About PEJ | Contact Us
 Powered by: Phase2 Technology
|
