Market Selection

Cities were selected by the research team based on Nielsen Media Research market rankings. Markets were grouped into four quartiles on the basis of the number of television households in each. Further adjustments were made to ensure large markets were not under-represented in the sample. Five markets were chosen within the top three quartiles and four markets in the fourth quartile were selected randomly, after stratification, to ensure geographic diversity.

Directed Market Selection and Rollovers

Two stations in each market quartile were selected for repeat study in 1999 to track their performance over time. Additional stations in each quartile were randomly selected, except in quartile two, where two new markets were selected purposively: Dallas representing a market in which at least one station is known to be attempting "up-market" appeal; Miami in which one station is reputedly going "down-market." Given the wide interest in both strategies these additions to the sample make the study more useful to news professionals.

Taping, Screening, and Inclusion

Research associates in each of the 19 markets taped newscasts during the following 1999 Monday-Friday time periods: February 15 - February 19 (sweeps primary) and February 22 - February 26 (sweeps secondary); April 12 - April 16 (non-sweeps primary) and April 19 - April 23 (non-sweeps secondary). For sweeps, primary days were used, unless unavailable because of preemption or taping error. In those cases, broadcasts from the secondary taping period were substituted, making every effort to match the appropriate day of the week. Only broadcasts from Monday, April 19 were used as substitutes for the non-sweeps period. The events in Littleton, Colorado occurred on Tuesday, April 20; using subsequent broadcasts as substitutes could have had unmanageable effects on the sample. No broadcasts from April 20 - 26 are included in this analysis. Where necessary, non-sweeps June dates were used as substitutes. (Note: for WFOR/Miami, taping error required the substitution of a June non-sweeps and a May sweeps week.)

Each half-hour broadcast was initially screened and precoded in its entirety by a single coder. The precoding process confirmed the date/timeslot of each broadcast and identified and timed individual stories. In accord with the instructions of the design team, recurring sports and weather spots were merely classified and timed; regular sports and weather segments were not part of any additional coding and are not reflected in any of the analysis or totals presented.

Story Coding and Scoring

Broadcasts were coded in their entirety by a single coder, via multiple story viewings. Working with a standardized codebook and coding rules, the process began with inventory variables, capturing information about broadcast date, market, station, network affiliation, etc. The second part of the coding scheme consisted of recordable variables, including story length, actors, and topics. The final section of the coding scheme contained the rateable variables. These were the measurements identified by the design team as quality indicators. The range in maximum possible points reflects the hier- archical significance of each value as per quantitative analysis of the design team's input. Each rateable variable was assigned both a code and a point score. Here are the variables and their maximum possible points per story: Focus, 10. Enterprise, 8. Source Expertise, 9. Balance Via Number of Sources, 5. Balance Via Viewpoints, 5. Sensationalism, 3. Story Understandibility, 2. Community Relevance, 8. The score-per-story represents points earned via rateable variables. To make the coding as objective as possible we did not try to rate how elegantly the story was organized.

Topic Diversity

Following the design team's directives, no story points were earned for topics; that is, no one topic was considered more important than another. Instead, the score-per-broadcast was calculated to reward stations for topic diversity, taking into account both the number of stories presented, and allowing for the additional minutes often added in post-prime timeslots or for the occasional broadcast where taping error occurred. For each news broadcast, a story: topic ratio was calculated by dividing the number of stories by the number of topics. That ratio was then converted to a broadcast multiplier. Next, the broadcast's scores-per-story were totaled, then divided by the number of stories, to reach an average score-per-story. The appropriate multiplier was then applied to the average score-per-story to reach the daily broadcast score. Finally, each station's 10 daily broadcast scores were totaled to reach the aggregate station score.

Intercoder Reliablity

Intercoder reliability measures the extent to which two coders, operating individually, reach the same coding decisions. For this project, the principal coding team comprised six people, who were trained as a group. One coder was designated as the control coder, and worked off-site for the duration of the project. At the completion of the general coding process, the four on-site coders, working alone and without access to the control coder's work, recoded 40% of the broadcasts completed by the control coder. Daily scores were found to be reliable within +/- 0.59 points per day, as per the comparative daily broadcast scores of general coders versus the control coder.

Ratings Analysis

The data collected, analyzed, and presented in this study are drawn from the Nielsen National Station Index (NSI) produced by Nielsen Media Research. The NSI measures television viewership for all U.S. television markets quarterly. The quarterly measurement periods are known as "Sweeps Months," and include February, May, July and November. The data presented in this report are based on Nielsen estimates of the weekday average household rating for each of 12 sweeps periods ranging from May, 1996 to February, 1999.
Ordinary least-squares regression was used to determine the slope (ratings trend over the three-year period) for each newscast, with time as the independent variable and rating as the dependent variable. It should be noted that trendline distribution reflects Windsorized data. Because of an inordinate amount of missing data, KPIX (San Francisco) and KTVA (Anchorage) were outliers and were removed from the analysis. The aggregate score was then matched with ratings information.