July 23, 2019

State of the News Media Methodology

The State of the News Media fact sheets use a range of different methodologies to study the health of the U.S. news industry, including custom analysis of news audience behavior, secondary analysis of industry data and direct reporting to solicit information unavailable elsewhere.

State of the News Media industry data

The State of the News Media fact sheets consist of data originally generated by other individuals or organizations that Pew Research Center then collected and aggregated.

For the data aggregated from other researchers, Pew Research Center’s team took several steps. First, Center researchers tried to determine what data had been collected and by whom for the media sectors studied. In many cases, this included securing rights to data through licensing fees or other means, and often included paying for use of the data.

Next, researchers studied the data closely to determine where elements reinforced each other and where there were apparent contradictions or gaps. In doing so, Pew Research Center endeavored to determine the value and validity of each dataset. That, in many cases, involved going back to the sources that collected the data in the first place. Where there were conflicts, the Center has included all relevant sources and tried to explain their differences, either in chart notes or in the text.

All sources are cited in chart and graphic notes or within the text of the report. Some data providers may update data for past and future years on an annual basis.

Analysis of Comscore television audience data

In 2018, Pew Research Center began using data from Comscore StationView Essentials® and TV Essentials® for our TV news audience analysis, and, as such, the current fact sheets include figures for the past three years. Local TV news audience data is from Comscore StationView Essentials®, while cable and network TV audience data is from Comscore TV Essentials®. Local TV audience data for 2016 and 2017 have been updated by Comscore to provide accurate trending for local TV that aligns with the national rollup, (i.e. how the average audience for local newscasts are rolled up into a figure for the U.S. overall).

Earlier versions of the three TV fact sheets included data stretching back to 2007 and used data from Nielsen Media Research. That data is available in the 2017 archived fact sheets. Because of differences between the two sets of data points, figures from prior years are not directly comparable with the Comscore data shown here.

Analysis of Comscore digital audience data

In order to provide as clear a sense as possible of the digital audience data of U.S. news organizations, researchers took several steps using digital audience measurement data from Comscore, a cross-platform measurement company.

After the initial selection process, each Comscore entry representing an outlet or website (“entity”) was individually vetted by researchers and unusual shifts in data over time were checked with Comscore analysts. Using feedback from analysts, entities that registered increases or declines in unique visitors over the time period studied (October to December of each year from 2014 to 2018) that were likely due to inorganic changes in measurement or site structure were considered to not have data that could be trended and were removed from the analysis.

For all outlets, entities that are representative of an entire web domain (“total domain entities”) were used whenever possible. When domain entities were incomplete – i.e., when they did not represent traffic to an entire web domain – custom entities that represent the total domain were used when available. Multiple entities of the same URL that were the result of a structure change in the Comscore database over time were considered comparable. Researchers consulted Comscore analysts throughout the entity selection process.

Selection

Newspapers: Researchers first assembled a list of the top 49 newspapers by average Sunday circulation for Q3 2015, Q3 2016, Q3 2017 and Q3 2018, according to the Alliance for Audited Media data. The Wall Street Journal was then added to the list, as it is one of the largest papers in the U.S., but does not have a Sunday edition, to produce a list of the top 50 newspapers by circulation. Each was matched with its associated total-domain entities in Comscore. Researchers then analyzed the Comscore data for October, November and December in each year. The following 50 entities were used in 2018:

AJC.COM
ARKANSASONLINE.COM
AZCENTRAL.COM
BALTIMORESUN.COM
BOSTONGLOBE.COM
BUFFALONEWS.COM
CHICAGOTRIBUNE.COM
CHRON.COM
CINCINNATI.COM
CLEVELAND.COM
COURANT.COM
DALLASNEWS.COM
DENVERPOST.COM
DISPATCH.COM
ELNUEVODIA.COM
FREEP.COM
INDYSTAR.COM
JSONLINE.COM
KANSASCITY.COM
LATIMES.COM
MERCURYNEWS.COM
MLIVE.COM
MYSANANTONIO.COM
NEWSDAY.COM
NJ.COM
NYDAILYNEWS.COM
NYPOST.COM
NYTIMES.COM
OCREGISTER.COM
OMAHA.COM
OREGONLIVE.COM
ORLANDOSENTINEL.COM
PHILLY.COM
PILOTONLINE.COM
POST-GAZETTE.COM
SACBEE.COM
SANDIEGOUNIONTRIBUNE.COM
SEATTLETIMES.COM
SFGATE.COM
STARADVERTISER.COM
STARTRIBUNE.COM
STLTODAY.COM
SUN-SENTINEL.COM
SUNTIMES.COM
SYRACUSE.COM
TAMPABAY.COM
TWINCITIES.COM
USATODAY.COM
WASHINGTONPOST.COM
WSJ.COM

 

Digital-native news outlets: Researchers assessed all domains from 11 Comscore categories (Business to Business, Directories/Resources, Entertainment, Games, Lifestyles, Multi-Category, News/Information, Services, Social Media, Sports and Technology) with at least 10 million average monthly unique digital visitors in the fourth quarter of 2018. From that set of entities, they then selected digital-native news outlets using the following criteria:

  1. Must be “born on the web,” i.e. not the website of a legacy news brand (note: may be owned by a legacy media company).
  2. It is a publisher of original content about news, defined as current events affecting public life (can include both original reporting and commentary/analysis). Sites are judged by an assessment of the material appearing on their homepage. A review of top stories on the homepage must render some evidence of original reporting, such as interviews, eyewitness accounts or referral to source documents, by a dedicated reporter/editorial staff. Sites are also judged as news publishers if they self-describe as an organization that produces news, either in the subject headers/navigation bar and/or in their “about” or advertising section through usage of terms like “news,” “journalism,” “covering” or “informing.”
  3. It is not entirely focused on reviews, advice, recipes or unedited raw data.
  4. It is not primarily a user-generated or aggregated content platform (such as Medium, Reddit or Wikipedia). Branded content such as NBA.com was also excluded.

The following 37 entities were used for 2018:

247SPORTS.COM
90MIN.COM
BGR.COM
BLEACHERREPORT.COM
BUSINESSINSIDER.COM
BUSTLE.COM
BUZZFEED.COM
BUZZFEEDNEWS.COM
CNET.COM
COMICBOOK.COM
DEADSPIN.COM
DIGITALTRENDS.COM
EATER.COM
ELITEDAILY.COM
ENGADGET.COM
GIZMODO.COM
HOLLYWOODLIFE.COM
HUFFINGTONPOST.COM
IGN.COM
INVESTOPEDIA.COM
MARKETWATCH.COM
MASHABLE.COM
MAXPREPS.COM
POLITICO.COM
POLYGON.COM
REFINERY29.COM
SBNATION.COM
SLATE.COM
TECHRADAR.COM
THEDAILYBEAST.COM
THEROOT.COM
THEVERGE.COM
THISISINSIDER.COM
THRILLIST.COM
TMZ.COM
UPROXX.COM
VOX.COM

A fresh cohort was chosen in each year using the criteria above. Audience information was calculated based on the cohort for that year to reflect the characteristics of the most popular digital-native sites at the time.

Analysis

For each website, minutes per visit and unique visitors for October to December of each year come from the Comscore Media Metrix® Multi-Platform U.S. database for Total Digital Population.

Comparisons year over year are between monthly averages of October to December data in each year.

For sites that didn’t meet the reporting threshold for one month out of a quarter, researchers averaged the two months for which data was available.

Digital-native news outlet audit

Researchers studied several outreach avenues that digital-native news outlets could take to engage with their audiences. For mobile apps, researchers searched the Google Play and iOS App Store for official apps from each outlet. For newsletters, researchers searched each outlet’s site for a sign-up form. For Apple News, researchers searched the Apple News app for official channels for the outlet. For Flipboard, researchers searched the Flipboard app and website for official pages for the outlet. For podcasts, researchers searched the iTunes podcast store for podcasts from the outlet and performed a search on each outlet’s site. An outlet was determined to support comments if at least one of the first five stories on its homepage supported comments at the time of analysis. For social media outreach, researchers searched for official pages, accounts or channels for the outlet on each platform, as well as on the outlet’s primary website.

Digital economic analysis

Economics data for the digital fact sheet comes from eMarketer’s U.S. Ad Spending estimates. Data from previous years is updated annually. In previous years, eMarketer included data about Amazon’s display ads. This data was not included in eMarketer’s 2018 display ads dataset, so Amazon is not broken out separately in any year for this analysis. For 2017 and 2018, eMarketer adjusted its format definitions, substantially changing the way the categories were structured. Because of these changes, data from 2016 and earlier is not included in this report but can be found in archived versions of the fact sheets.

Employment and wage data

Employment and wage trends are based on data from the Bureau of Labor Statistics (BLS) Occupational Employment Statistics (OES) survey. Pew Research Center uses the OES data to estimate employment and median wages for four media-related occupations in five information-producing industries. The OES survey produces annual estimates of occupational employment and wage rates for full- and part-time wage and salary workers (excluding the self-employed) in nonfarm establishments.

The Center’s analyses focus on five industries: 1) newspaper publishers, 2) television broadcasting, 3) cable and other subscription programming, 4) radio broadcasting and 5) other information services, which is the best match for digital-native publishers. (See below for more information on this category.)

Within each industry, the number of newsroom employees is limited to four occupations associated with news production: 1) news analysts, reporters and correspondents; 2) editors; 3) photographers (e.g., photojournalists) and 4) television, video, and motion picture camera operators and editors (e.g., news videographers, television news video editors).1 This allows a focus on newsroom staff rather than those on the business or distribution side, such as advertising sales agents, printing press operators and delivery truck drivers. (It does not allow for the inclusion of layout artists, designers or digital producers, as there are no occupation codes for employees doing this work specifically in the group of media and communication equipment workers.)

The OES survey produces employment and wage estimates annually for over 800 occupations. National occupational estimates for specific industries are also available. The wage data is adjusted for inflation using annual averages from the BLS Consumer Price Index Research Series (CPI-U-RS) with the latest year in the trend as the base year.

Using this single source of data allows for comparable employment estimates across the industry groups by standardizing the occupations included, rather than relying on estimates that are either specific to certain sectors of the news media industry or are not produced on an annual basis.

OES data, however, does carry some limitations. For example, OES uses the federal government’s Standard Occupational Classification system, making the occupations for which they have wage data not quite as fine-grained or industry-specific as the salary data published by private sources, such as the Radio Television Digital News Association (RTDNA), which uses a classification system tailored to include specific occupations in the television and radio industries, such as weathercaster, sports anchor or tape editor.

Additionally, OES uses the federal government’s North American Industry Classification System (NAICS), which subsumes the category most closely corresponding to digital-native publishers – internet publishing and broadcasting and web search portals – under the broader other information services industry, and only allows the data to be broken out for this umbrella category.

The estimates of this broader industry still reflect the employment and wage trends of digital-native publishing for two reasons. First, internet publishing and broadcasting and web search portals make up about three-quarters (79%) of the other information services industry, according to the 2018 Current Employment Statistics survey. Second, it has represented 60% or more of the other information services industry since 2008.

However, the number of digital-native newsroom employees over time may be slightly elevated, as it includes newsroom employees from other categories in the broader other information services industry, such as those from the news syndicates category. It also may include legacy news organizations that are now published exclusively on the web. Because of these limitations, and given that digital news is a rapidly evolving industry, the most valuable information derived from this data is not annual employment and wage levels, but trends over time.

Note that the employment data for the other information services industry is shown starting from 2008. This is because OES switched from the 2002 NAICS to the 2007 NAICS beginning with the May 2008 estimates, and the other information services industry is defined differently under the two systems. Data for the newspaper, radio, television and cable industries were not similarly affected. Additionally, OES may withhold from publication employment and wage data for some occupations due to, for example, failure to meet Bureau of Labor Statistics quality standards or the need to protect the confidentiality of survey respondents. Gaps in the trend data presented reflect these limitations in the OES data.

The OES survey produces annual estimates by combining six panels of data collected over a three-year period. Every year, two new panels of data are added, and the two oldest panels are dropped, resulting in a moving average staffing pattern. The three years of employment data is benchmarked to represent the total employment for the reference period. Because annual estimates from overlapping three-year periods are based on nearly the same data, it is difficult to make conclusive year-to-year comparisons. Comparisons are best made between non-overlapping periods. OES data is not designed for making comparisons through time, and such comparisons should be interpreted with caution.

Acknowledgments

Assistance in data analysis was provided by Nami Sumida, Hadeel Saab and Amy Shim. Sara Atske, Selena Qian and Peter Bell provided web producing and graphic support, and Aleksandra Sandstrom and David Kent provided copy editing support, while Hannah Klein, Rachel Weisel and Calvin Jordan provided communication support.