Local TV News Project 2001

The study this year examined broadcast news programs in 14 cities, 43 stations in all. We also studied the three broadcast network evening news programs. Taping occurred during a February sweeps week and an April non-sweeps week. A team of professional coders analyzed 6,472 stories from 470 broadcasts, or 235 hours of local news. The results were then statistically analyzed by researchers Princeton Survey Research Associates and at Wellesley College and interpreted by a team of journalists.

Our definition of quality is the same established by our design team of local TV news professionals. We stress the basics: a newscast should cover a broad range of topics, focus on the significant aspects of the news, be based on original reporting, provide credible information, use multiple sources, balance stories with multiple points of view, and contain locally relevant stories. We continue to use the system developed by separate teams of university scholars and professional researchers to grade newscasts by a point system matched to these criteria. As in years past presentation is a very minor factor. So that grading can be accomplished objectively, stories score well based on an accumulation of the simple journalistic values mentioned above.

This year's study also included a national mail survey of news directors, conducted between June and August 2001. A random sample of 196 news directors was selected from an enumerated list of television stations. One hundred eighteen news directors completed the surveys for a response rate of 60 percent. The sample of 118 respondents represent 107, or more than half, of the 210 local television markets throughout the country that produce news. Results are therefore reported unweighted. The survey has a margin of error of plus or minus 5 percent, which means statistically that in 95 samples out of 100 the results will not differ more than 5 percent from those reported here.

Taping, Screening, and Inclusion

Research associates in the designated markets taped newscasts for the following 2001 Monday-Friday time periods: February 12 - February 16 (sweeps primary) and February 19 - February 23 (sweeps secondary); April 9 - April 13 (non-sweeps primary) and April 16 - April 20 (non-sweeps secondary). For both monitoring periods, primary days were used, unless unavailable due to preemption or taping error. In those cases, broadcasts from the secondary taping period were substituted, making every effort to match the appropriate day of the week.

Note: the following substitutions were necessary due to taping error.

KSL/Salt Lake City April 4 (non-sweeps) WCAU/Philadelphia April 5 (non-sweeps) KULR/Billings April 30, June 25 (non-sweeps) KTVQ/Billings June 25 (non-sweeps)

Precoding

Each broadcast was initially screened and precoded in its entirety by a single coder. The precoding process confirmed the date/timeslot of each broadcast and identified and timed individual stories. Per the instructions of the design team, recurring sports and weather spots were merely classified; regular sports and weather segments were not part of any additional coding and are not reflected in any of the analysis or totals presented in this study.

Story Coding and Scoring

Broadcasts were coded in their entirety by a second coder, via multiple story viewings. Working with a standardized codebook and coding rules, the process began with inventory variables, capturing information about broadcast date, market, station, network affiliation, etc. The second part of the coding scheme consisted of recordable variables, including story length, actors, and topics. The final section of the coding scheme contained the rateable variables. These were the measurements identified by the design team as quality indicators. The range in maximum possible points reflects the hierarchical significance each value as per quantitative analysis of the design team's input. Each rateable variable was assigned both a code and a point score. The rateable variables and their maximum possible points per story are as follows: Focus, 10. Enterprise, 8. Source Expertise, 9. Balance Via # of Sources, 5. Balance Via Viewpoints, 5. Sensationalism, 3. Presentation, 2.Community Relevance, 8. The score-per-story represents points earned via rateable variables.

Topic Diversity

Per the design team's directives, no story points were earned for topics; that is, no one topic was considered more important than another. Instead, the score-per-broadcast was calculated to reward stations for topic diversity, taking into account both the number of stories presented, and allowing for the additional minutes often added in post-prime timeslots or for the occasional broadcast where taping error occurred. For each news broadcast, a story:topic ratio was calculated by dividing the number of stories by the number of topics. The story:topic ratio was then converted to a broadcast multiplier. Broadcast scores-per-story were totaled, then divided by the number of stories, to reach an average score-per-story. The appropriate multiplier was then applied to the average score-per-story to reach the daily broadcast score. Finally, each station's 10 daily broadcast scores were totaled to reach the aggregate station score.

Intercoder Reliability

Intercoder reliability measures the extent to which two coders, operating individually, reach the same coding decisions. For this project, the principal coding team was comprised of 5 individuals, who were trained as a group, augmented by 2 precoders. One coder was designated as the control coder, and worked off-site for the duration of the project. At the completion of the general coding process, the on-site coders, working alone and without access to the control coder's work, recoded 33% of the broadcasts completed by the control coder. Daily scores were found to be reliable within +/- 0.74 points per day, as per the comparative daily broadcast scores of general coders vs. the control coder.

Ratings Analysis

The data collected, analyzed, and presented in this study are drawn from the Nielsen National Station Index (NSI) produced by Nielsen Media Research. The NSI measures television viewership for all U.S. television markets quarterly. The quarterly measurement periods are known as "Sweeps Months," and include February, May, July and November. The data presented in this report are based on Nielsen estimates of the weekday average ratings for each of 12 sweeps periods ranging from May, 1998 to February, 2001 for 43 local news telecasts in 14 Designated Market Areas (DMAs).

Ratings were collected for the following demographics: All TV Households, Women 18-49, Women 25-54, Men 18-49 and Men 25-54. Share data was also collected for all TV Households. As a supplement to the 43 stations in the main study, an additional sample of 113 stations from various markets were analyzed separately according to the same procedure employed for the main study. The procedure is as follows:

Ordinary least-squares regression was used to determine the slope (ratings or share trend over the three-year period for each demographic category) for each newscast, with time as the independent variable and rating or share as the dependent variable. From the distribution of slopes, a five-point coding scheme was used to assign a trend value for each newscast. The five-point scheme is curved, and is based on the mean and standard deviation of the sample. For each telecast, ratings data were obtained for the 1/2 hour period prior to each newscast. These data were used to determine the trend in lead-in retention for each newscast.

Lead-in retention is defined as the percentage of the rating value attained by the newscast as compared to the lead-in period. For example, if a newscast received a household rating of 8 while the lead-in half hour received a rating of 10, this would be calculated as 80% retention. Ordinary least-squares regression was used to determine the slope (percent retention trend over the three-year period) for each newscast, with time as the independent variable and lead-in retention (percentage) as the dependent variable. This type of analysis is a fairer way of assessing lead-in retention than an a simple average as it is not influenced by the potential magnitude in ratings difference between the lead-in program and the newscast. From the distribution of slopes, a five-point coding scheme was employed in a method similar to those for the ratings trends. Comparisons of trend rankings were then made by demographics, lead-in to target newscast, and a differential of share-rating trend.