September 16, 2013

How Al Jazeera Tackled the Crisis Over Syria

Methodology

About this Study

A number of people at the Pew Research Center’s Project for Excellence in Journalism worked on Pew Research’s “The Crisis over Syria: How Al Jazeera America Tackled Its First Major Story.” Associate Director Mark Jurkowitz and Director Amy Mitchell wrote the report. Research Analyst Katerina-Eva Matsa analyzed the statistics and created the charts. Coding and data analysis were conducted by Researchers Monica Anderson, Paul Hitlin, Katerina-Eva Matsa and Nancy Vogt. Jesse Holcomb copy edited the report. Dana Page handles communications for the project.

Methodology

This special report by the Pew Research Center’s Project for Excellence in Journalism examined news media coverage of Syria from August 26 through August 31, 2013.

Data regarding the coverage of Syria in the main cable news media sample were derived from the Pew Research Center’s in-house coding operation.

Human Coding of News Media

Sample Design

The sample of mainstream cable news included the following programs from a total of 5 channels, which followed the schedule listed below.

Cable TV (14 in all, Monday-Friday)

Morning (6:00 to 7:00 a.m.) coded one hour of programming every weekday
CNN
Fox News
MSNBC

BBC

Morning (7:00 to 8:00 a.m.) coded one hour of programming every weekday
Al Jazeera

Nighttime CNN – coded one hour of the following every day

Situation Room (6 p.m.)
Anderson Cooper 360

Nighttime Fox News – coded one hour of the following every day

Fox Report w/ Shepard Smith
O’Reilly Factor

*On August 30, 2013 Bill O’Reilly did not host his show. Instead, Greta Van Susteren was on. Researchers still coded this segment.

Nighttime MSNBC – coded one hour of the following every day

Hardball (7 p.m.)
The Rachel Maddow Show

*On August 30, 2013 the Rachel Maddow show did not air. Instead, Hardball was repeated. Therefore researchers did not code this segment.

Nighttime Al Jazeera – coded one hour of the following every day

America Tonight (9 p.m.)

Consider This (10 p.m.)

Nighttime BBC America – coded  a half hour of the following every day

BBC World News America (5:30 p.m.)

Cable TV (4 in all, Saturday)

Daytime (2:00 to 3:00 p.m.) coded one hour of programming after the Obama address
CNN
Fox News
MSNBC

Al Jazeera

Story Selection

Stories were included in the study if 50% or more of their content was on Syria. To find the relevant stories in all outlets researchers watched the full hour of programming. Then, researchers read or watched each story to see if the 50% of the story or more was on Syria. Commercials, promos, and teasers of upcoming stories were all excluded from the sample.

This process resulted in a sample of 327 stories.

Human Coding of Stories

The data in this study were created by a team of four experienced coders.

The unit of analysis for this study was the story. If a story contained two or more frames, the frame that was given the most attention (in terms of seconds) in the story was assigned.

Each story was coded for the messages it conveyed.  While watching a story, coders checked all the messages that appeared about Syria. Direct and indirect quotes were counted along with assertions made by journalists themselves.

Intercoder Testing

We have conducted intercoder testing on all of the housekeeping variables (such as source, date, opinion vs. reporting and format) on numerous occasions, and reached a level of agreement of 80% or higher on each of them.

In addition to the intercoder testing conducted on all housekeeping variables, supplemental testing was conducted on the additional variables specific to this study. For the following variables, 20 randomly selected stories were coded by all members of the coding team in two rounds -10 stories were coded in the beginning of the study and 10 in the end.

The percent of agreement for each variable was as follows, which was derived from the final testing:

  • Frame: 87.5%
  • All sources combined: 98%

 

Individual messages:

  • U.S./Allies Should Get Involved Militarily in Syria: 90%
  • U.S./Allies Should Not Get Involved in Syria: 87.5%
  • Obama’s Handling of the Situation Is Poor: 100%
  • U.S./Allies Do Not Understand Mideast – 100%
  • There Are No Good Choices: 97.5%
  • Syrians Want U.S. to Get Involved: 100%
  • Syrians Do Not Want U.S. to Get Involved: 100%
  • U.S. Conflict Will Lead to Larger Middle East Problems: 90%
  • Other Countries Will Make It Hard for U.S.:93%
  • U.S. Should Go to UN/Congress First: 95%
  • Involvement Would Last Longer: 97.5%
  • Iraq Experience Should Make Us Cautious about Intel: 97.5%