New Media Index Methodology

Overview

The New Media Index is a weekly report that captures the leading commentary of blogs and social media sites focused on news and compares those subjects to coverage in the mainstream press.

The New Media Index is a companion to PEJ’s weekly News Coverage Index. Blogs and other new media are an important part of creating today’s news information narrative and in shaping the way Americans interact with the news. The expansion of online blogs and other social media sites has allowed news-consumers and others outside the mainstream press to have more of a role in agenda setting, dissemination and interpretation. PEJ aims to find out what subjects in the national news the online sites focus on, and how that compares with the narrative in the traditional press.

Universe

Two prominent Web tracking sites, Technorati and Icerocket, monitor more than 100 million blogs and over 250 million pieces of social media, using the links to articles embedded on these sites as a proxy for determining what these subjects are.

Each of these two sites offers lists of the most linked-to news stories, based on the number of blogs or other sites that link to them. These stories are almost always from mainstream news sources such as the New York Times Web site or CNN.com.

Each weekday between 9 and 10 am ET, a PEJ staff member manually captures these lists.  

From those lists, the top five linked-to articles are captured for further analysis by PEJ staff. That results in ten articles a day and 50 articles a week.

Coding Procedures

Once the lists of articles are compiled, PEJ staff conducts a content analysis of the subject matter of these mainstream news articles in a similar manner to the News Coverage Index.

Almost all of the codes and rules are the same as with the NCI. For details about the variables and PEJ’s intercoder testing procedures for these codes, refer to the detailed methodology about the News Coverage Index.

Among the variables is what we call Big Story, a variable also used in the NCI, which is a continuously evolving list of the major news storylines of the day. The Big Story variable is the primary one used to see which topics are getting the most attention. A story must be 50% about a particular topic in order to be given a specific Big Story code.

The only additional variables used in the NMI are identifying the original outlet of the news story and tracking the number of links aimed at each story included in the sample. Technorati and Icerocket provide the number of links within their lists.

Calculations

The priorities of the bloggers are measured in terms of percentage of links. Each time a news blog or social media Web page adds a link to its site directing its readers to a news story, it suggests that the author places at least some importance on the content of that article. The user may or may not agree with the contents of the article, but they feel it is important enough to draw the reader’s attention to it.

The calculations for the NCI have a different base. That Index uses the percent of time (in seconds) or space (in words) to arrive at the percent of newshole devoted to each topic.

The reason that the New Media Index uses a different measure, links rather than newshole, is because the nature of online media is different from other traditional forms of media. First, there is no limit to the amount of space that can be devoted to a specific story. In a newspaper, there is a limited amount of space on a front page, for example, and a television newscast is limited by its allotted amount of time. Web sites have no such limits.

Second, PEJ determined that in this procedure, the number of blogs that link to a news article are a far greater measure of the significance of that article online than the length of the story. A particular article might be quite long in terms of number of words, but if only a few blogs link to it, that article would have only a small influence in the new media environment. A short story that gets linked to many times has a far greater influence.

The percent of links for each Big Story is determined by taking the total number of links in the sample (from all 50 stories in the week) and then dividing that number by the number of links devoted to each specific Big Story. The percentages are then ranked in order to discover the five storylines that were most present in online commentary.

Differences from the NCI

In addition to the base calculation, there are three differences between the NMI and the NCI to note: 

1. While the capture times for the Web sites included in the News Coverage Index rotate each day, a decision was made to keep the times the same for the New Media Index. The reasoning is that since these lists compile the number of links to stories over a 48-hour window, rotating the time of capture would result in different increments of times between each capture. Through testing, PEJ has discovered that the stories on the lists change significantly more over a 24-hour period than they do over a 12 or 16-hour period.

2. While the News Coverage Index is comprised of primarily U.S.-based media outlets, the aggregators of blogs and other social media include both U.S. and non-U.S. blogs. In addition, stories that are linked to can be from non-U.S. sources. However, according to PEJ’s research over the last two months, the only non-U.S. news stories included in the top lists for Technorati and Icerocket have been the BBC (whose Web site is part of the News Coverage Index) and the Guardian.

3. PEJ’s weekly News Coverage Index includes Sunday newspapers while the New Media Index is Monday through Friday.

YouTube Videos

The New Media Index also includes a section of the most popular news video on YouTube each week.  

Each Friday at noon ET, a PEJ staff member captures the list of most viewed news and politics videos on YouTube over the previous week. These videos are categorized as such on the YouTube site and are often a mix of mainstream news reports, raw footage relating to breaking events, or other types of public affairs clips. PEJ determines the top five most viewed videos as they are listed on YouTube’s page at the time of capture.

Note: After consulting various reference guides and outside consultants on usage, the Project has chosen to refer to its several weekly content analysis reports as “indexes”—the version largely accepted in journalism—instead of “indices”—a term used more frequently in scientific or academic writing.