Journalism and Research Again

Lots of Twitter and blog activity in response to NYT article about Chetty, Friedman, and Rockoff research paper on effects of teachers on students' lives.

No small amount of the commentary is about how when journalists pick "interesting" bits out of research reports to construct a "story" they often create big distortions in the social knowledge-base.

So what can reporters do when trying to explain the significance of new research, without getting trapped by a poorly-supported sound bite?

Sherman Dorn has an excellent post on the case, "When reporters use (s)extrapolation as sound bites," that ends with some advice:

1. "If a claim could be removed from the paper without affecting the other parts, it is more likely to be a poorly-justified (s)implification/(s)extrapolation than something that connects tightly with the rest of the paper."
2. "If a claim is several orders of magnitude larger than the data used for the paper (e.g., taking data on a few schools or a district to make claims about state policy or lifetime income), don’t just reprint it. Give readers a way to understand the likelihood of that claim being unjustified (s)extrapolation."
3. "More generally, if a claim sounds like something from Freakonomics, hunt for a researcher who has a critical view before putting it in a story."

See also Matthew Di Carlo on ShankerBlog, Bruce Baker on SchoolFinance 101, and Cedar Reiner on Cedar's Digest