July 25, 2018

State of the News Media Methodology

The State of the News Media fact sheets use a range of different methodologies to study the health of the U.S. news industry, including custom analysis of news audience behavior, secondary analysis of industry data and direct reporting to solicit information unavailable elsewhere.

State of the News Media industry data

The State of the News Media fact sheets consist of data originally generated by other individuals or organizations that Pew Research Center then collected and aggregated.

For the data aggregated from other researchers, Pew Research Center’s team took several steps. First, Center researchers tried to determine what data had been collected and by whom for the media sectors studied. In many cases, this included securing rights to data through license fees or other means, and often included paying for use of the data.

Next, we studied the data closely to determine where elements reinforced each other and where there were apparent contradictions or gaps. In doing so, Pew Research Center endeavored to determine the value and validity of each dataset. That, in many cases, involved going back to the sources that collected the research in the first place. Where data conflicted, the Center has included all relevant sources and tried to explain their differences, either in footnotes or in the narratives.

All sources are cited in footnotes or within the narrative and in all graphics in the report.

Analysis of comScore digital audience data

In order to provide as clear a sense as possible of the digital audience data of U.S. news organizations, researchers took several steps using digital audience measurement data from comScore, a cross-platform measurement company.

After the initial selection process, each comScore entry representing an outlet or website (“entity”) was individually vetted by researchers and unusual shifts in data over time were checked with comScore analysts. Using feedback from analysts, entities that registered increases or declines in unique visitors over the time period studied (October 2014-December 2017) that were likely due to inorganic changes in measurement or site structure were considered to not have trendable data and were removed from the analysis.

For all outlets, entities that are representative of an entire web domain (“total domain entities”) were used whenever possible. When domain entities were incomplete – i.e., when they did not represent traffic to an entire web domain – custom entities that represent the total domain were used when available. Multiple entities of the same URL that were the result of a structure change in the comScore database over time were considered comparable. Researchers consulted comScore analysts throughout the entity selection process.

Selection

Newspapers: Researchers first assembled a list of the top 49 newspapers by average Sunday circulation for Q3 2015, Q3 2016 and Q3 2017 according to the Alliance for Audited Media data. The Wall Street Journal was then added to the list, as it is one of the largest papers in the U.S. but does not have a Sunday edition, to produce a list of the top 50 newspapers by circulation. Each was matched with its associated total-domain entities in comScore. Researchers then analyzed the comScore data for October, November and December in each year. The following 50 entities were used in 2017:

AJC.COM
ARKANSASONLINE.COM
AZCENTRAL.COM
BALTIMORESUN.COM
BOSTONGLOBE.COM
BUFFALONEWS.COM
CHICAGOTRIBUNE.COM
CHRON.COM
CINCINNATI.COM
CLEVELAND.COM
COURANT.COM
COURIER-JOURNAL.COM
DALLASNEWS.COM
DENVERPOST.COM
DISPATCH.COM
ELNUEVODIA.COM
FREEP.COM
INDYSTAR.COM
JSONLINE.COM
KANSASCITY.COM
LATIMES.COM
MERCURYNEWS.COM
MYSANANTONIO.COM
NEWSDAY.COM
NJ.COM
NYDAILYNEWS.COM
NYPOST.COM
NYTIMES.COM
OCREGISTER.COM
OKLAHOMAN.COM
OREGONLIVE.COM
ORLANDOSENTINEL.COM
PHILLY.COM
PILOTONLINE.COM
POST-GAZETTE.COM
SACBEE.COM
SANDIEGOUNIONTRIBUNE.COM
SEATTLETIMES.COM
SFGATE.COM
STARADVERTISER.COM
STAR-TELEGRAM.COM
STARTRIBUNE.COM
STLTODAY.COM
SUN-SENTINEL.COM
SUNTIMES.COM
TAMPABAY.COM
TWINCITIES.COM
USATODAY.COM
WASHINGTONPOST.COM
WSJ.COM

 

Digital-native news outlets: Researchers assessed all domains from 11 comScore categories (Business/Finance, Directories/Resources, Entertainment, Games, Lifestyles, News/Information, Regional/Local, Services, Social Media, Sports and Technology) with at least 10 million average monthly unique digital visitors in the fourth quarter of 2017. From that set of entities, they then selected digital-native news outlets using the following criteria:

  1. Must be “born on the web,” i.e. not the website of a legacy news brand (Note: may be owned by a legacy media company).
  2. It is a publisher of original content about news, defined as current events affecting public life (can include both original reporting and commentary/analysis). Sites are judged by an assessment of the material appearing on their home page. A review of top stories on the home page must render some evidence of original reporting, such as interviews, eyewitness accounts or referral to source documents, by a dedicated reporter/editorial staff. Sites are also judged as news publishers if they self-describe as an organization that produces news, either in the subject headers/navigation bar and/or in their “about” or advertising section through usage of terms like “news,” “journalism,” “covering” or “informing.”
  3. It is not entirely focused on reviews, advice, recipes or unedited raw data.
  4. It is not primarily a user-generated or aggregated content platform (such as Medium, Reddit or Wikipedia). Branded content such as NBA.com was also excluded.

The following 35 entities were used for 2017:

12UP.COM
247SPORTS.COM
BGR.COM
BLEACHERREPORT.COM
BREITBART.COM
BUSINESSINSIDER.COM
BUSTLE.COM
BUZZFEED.COM
CNET.COM
DEADSPIN.COM
DIGITALTRENDS.COM
EATER.COM
ELITEDAILY.COM
GAMESPOT.COM
GIZMODO.COM
HUFFINGTONPOST.COM
IBTIMES.COM
IGN.COM
INVESTOPEDIA.COM
JEZEBEL.COM
MASHABLE.COM
MAXPREPS.COM
POLITICO.COM
QZ.COM
REFINERY29.COM
SBNATION.COM
SLATE.COM
THEDAILYBEAST.COM
THEVERGE.COM
THRILLIST.COM
TMZ.COM
TOPIX.NET
UPROXX.COM
UPWORTHY.COM
VOX.COM

A fresh cohort was chosen in each year using the criteria above. Audience information was calculated based on the cohort for that year to reflect the characteristics of the most popular digital-native sites at the time.

Analysis

For each website, minutes per visit and unique visitors for October-December of each year were acquired from the comScore Media Metrix database for Total Digital Population.

Comparisons year over year are between monthly averages of October-December data in each year.

For sites that didn’t meet the reporting threshold for one month out of a quarter, we averaged across the two months for which we had data.

Digital-native news outlet audit

Researchers studied several outreach avenues that digital-native news outlets could take to engage with their audiences. For mobile apps, researchers searched the Google Play and iOS App Store for official apps from each outlet. For newsletters, researchers searched each outlet’s site for a sign-up form. For Apple News, researchers searched the Apple News app for official channels for the outlet. For podcasts, researchers searched the iTunes podcast store for podcasts from the outlet and performed a search on each outlet’s site. An outlet was determined to support comments if at least one of the first five stories on its homepage supported comments at the time of analysis. For social media outreach, researchers searched for official pages, accounts or channels for the outlet on each platform, as well as on the outlet’s primary website.

Employment and wage data

Fact sheets released after May 31, 2018, include employment and wage trends based on data from the Bureau of Labor Statistics (BLS) Occupational Employment Statistics (OES) survey. We use the OES data to estimate employment and median wages for four media-related occupations in five information-producing industries. The OES survey produces annual estimates of occupational employment and wage rates for full- and part-time wage and salary workers (excluding the self-employed) in nonfarm establishments.

The Center’s analyses focus on five industries: 1) newspaper publishers, 2) television broadcasting, 3) cable and other subscription programming, 4) radio broadcasting and 5) other information services, which is the best match for digital-native publishers. (See below for more information on this category.)

Within each industry, the number of newsroom employees is limited to four occupations associated with news production: 1) news analysts, reporters and correspondents; 2) editors; 3) photographers (e.g., photojournalists) and 4) television, video, and motion picture camera operators and editors (e.g., news videographers, television news video editors).1 This allows a focus on newsroom staff rather than those on the business or distribution side, such as advertising sales agents, printing press operators and delivery truck drivers. (It does not allow for the inclusion of layout artists, designers or digital producers, as there are no occupation codes for employees doing this work specifically in the group of media and communication equipment workers.)

The OES survey produces employment and wage estimates annually for over 800 occupations. National occupational estimates for specific industries are also available. The wage data presented are adjusted for inflation, using annual averages from the BLS Consumer Price Index Research Series (CPI-U-RS), with the latest year in the trend as the base year.

Using this single source of data allows for comparable employment estimates across the industry groups by standardizing the occupations included rather than relying on estimates that are either specific to certain sectors of the news media industry or are not produced on an annual basis.

Few other surveys produce estimates of newsroom employees, but the OES data closely tracks those sources that are available. When compared with the annual census of full-time newspaper journalists performed by the American Society of News Editors (ASNE), the employment estimates from OES for the newspaper publishers industry are slightly higher – in part due to the inclusion of part-time employees in the OES data – but follow the same general trend.

The OES employment estimates for the television broadcasting industry also closely follow estimates of the number of local television station newsroom employees published by the Radio Television Digital News Association (RTDNA).

No current, publicly available surveys provide comparable employment statistics for the cable, radio or digital news industries.

OES data, however, does naturally carry some limitations. For example, OES uses the federal government’s Standard Occupational Classification system, making the occupations for which they have wage data not quite as fine-grained or industry-specific as the salary data published by RTDNA, which uses a classification system tailored to include specific occupations in the television and radio industries, such as weathercaster, sports anchor or tape editor.

Additionally, OES uses the federal government’s North American Industry Classification System (NAICS), which subsumes the category most closely corresponding to digital-native publishers – internet publishing and broadcasting and web search portals – under the broader other information services industry, and only allows the data to be broken out for this umbrella category.

We believe the estimates of this broader industry still reflect the employment and wage trends of digital-native publishing for two reasons. First, internet publishing and broadcasting and web search portals make up about three-quarters (78%) of the other information services industry, according to the 2017 Current Employment Statistics survey. Second, it has represented 60% or more of the other information services industry since 2008.

However, the number of digital-native newsroom employees over time may be slightly elevated as they include newsroom employees from other categories in the broader other information sources industry, such as those from the news syndicates category. It also may include legacy news organizations that are now published exclusively on the web. Because of these limitations, and given that digital news is a rapidly-evolving industry, the most valuable information derived from these data are not annual employment and wage levels but trends over time.

Note that we show the employment data for the other information services industry starting from 2008. This is because OES switched from the 2002 NAICS to the 2007 NAICS beginning with the May 2008 estimates, and the other information services industry is defined differently under the two systems. Data for the newspaper, radio, television and cable industries were not similarly affected. Additionally, OES may withhold from publication employment and wage data for some occupations due to, for example, failure to meet Bureau of Labor Statistics quality standards or the need to protect the confidentiality of survey respondents. Gaps in the trend data we present reflect these limitations in the OES data.

The OES survey produces annual estimates by combining six panels of data collected over a three-year period. Every year, two new panels of data are added, and the two oldest panels are dropped, resulting in a moving average staffing pattern. The three years of employment data are benchmarked to represent the total employment for the reference period. Because annual estimates from overlapping three-year periods are based on nearly the same data, it is difficult to make conclusive year-to-year comparisons. Comparisons are best made between non-overlapping periods. OES data are not designed for making comparisons through time, and such comparisons should be interpreted with caution.

Acknowledgments

In addition to those who compiled the fact sheets, Elizabeth Grieco developed the employment methodology. Assistance in data analysis was provided by Sophia Fedeli, Nami Sumida and Mason Walker. Ariana Rodriguez-Gitler, Sara Atske and Margaret Porteus provided web producing and graphic support, while Hannah Klein and Rachel Weisel provided communication support.

  1. In applying this group of occupations across different sectors of the news media, we make the assumption that this fourth category – television, video and motion picture camera operators and editors – includes people working as web video producers for websites and other digital products of newspapers or radio broadcasters.