Wednesday, October 03, 2007

Greenhouse gases from biofuels and the reporting of science

You may have read that a study by Nobel Prize winner Paul Crutzen and colleagues currently under review reports that biofuels, like corn-base ethanol and rapeseed biodiesel, emit more greenhouse gases than the fossil fuels they replace. It was covered by Reuters, the Times (UK) and the magazine Chemistry World (update: podcasts available on Scitizen). The news then bounced all over the internet, appearing in Grist, Green Car Congress, and a number of blogs like Alternapower, Green Diary, Biofuels Digest, Climateer Investing, Earth2Tech, Big Biofuels Blog, Classically liberal, After Gutenberg, Digital Journal, I could go on.

The paper, submitted to Atmospheric Chemistry and Physics, uses a global budget of the sources and sinks of atmospheric greenhouse gas nitrous oxide (N2O), which originates in part from nitrogen fertilizer use, to estimate a rate at which N2O is emitted in the production of biofuel crops like corn, rapeseed (canola) and sugarcane. N2O is a much less common greenhouse gas than CO2, but has each molecule has a greater “warming” effect. The authors then contrast the global warming potential of N2O in growing the biofuel crop with that of the CO2 saved by replacing gasoline use. Their results suggest that corn ethanol and rapeseed biodiesel would lead to a net “warming” from the N2O emissions alone.

The problem is, as I was quoted as saying in the Chemistry World article, is that their method and their results are probably wrong. The paper is still under review and many of the comments on the paper, including my own, disagree with both their methods and their conclusions about a net “warming” from N2O emissions.

Let me be clear before continuing: this is not to say that the production and use of biofuels like corn ethanol will certainly emit far fewer greenhouse gases than the production and use of regular gasoline. A number of studies (Farrell et al, 2005) have concluded that when you include the entire production cycle, corn-based ethanol is either a small “win” or possibly a small loss. Either way, these other studies are including all sources of emissions during production, including operating the machinery, producing the fertilizer, processing the grain, and the N2O from fertilizer use. The Crutzen et al. paper reports that the N2O emissions from fertilizer alone makes corn-based ethanol and rapeseed net losers in the emissions reduction game.

Here’s the central science problem: The paper’s global budget analysis leads to the conclusion that 3-5% of nitrogen fertilizer is eventually emitted to the atmosphere as N2O, more than twice the rate of 1-2% found in all previous research. There is no actual physical evidence for the 3-5% result. The budget analysis used to determine that figure is interesting and clever, but it is fraught with problems (for more, I refer you to the comments). The take home message is that if you substitute the lower value, as most of us would, the basic conclusions of the study change.

The larger problem with this story has less to do with the science than with the reporting of science. You no doubt noticed one key phrase in the leading paragraph of this post. Nobel Prize Winner. That jumps out, no? The revered status implies this is research we should trust.

There is one other phrase you are unlikely to have noticed above. Under review. You see, the study in question has not been accepted for publication. It is under review with the journal Atmospheric Chemistry and Physics. That is important. It means that the journal and the reviewers, ostensibly assigned as representatives the scientific community, are deciding whether the methods are appropriate, whether the results can be falsified, whether the conclusions are sensible and whether the article as a whole is worthy of presentation in its submitted form.

The review process is key to science. It filters out fatally flawed research and helps authors improve a questionable paper by providing the judgment of outside experts. There’s no shame in reviews. All of us, whether graduate student or Nobel Prize winner, need them. Safe to say every paper I’ve published is better because of the input of anonymous peer reviewers.

In this case, that process is still underway. But that's where things get screwy. Unlike other journals, where submitted articles are sent to individual expert reviewers selected by the editors, APC includes an open review process. That means in addition to the normal peer review, anyone in the community can read the submitted paper online and offer comments. Personally, I like the system, as it gives authors a wider array of reviews and helps eliminate the chances that one irrational reviewer will derail a good piece of research.

The unintended consequence of the open discussion, however, is that an unpublished, and hence, unfinished paper is there for anyone to read. The paper can be reported in the media and the public can get the mistaken that the findings are accepted by the scientific community.

Science reporting is often a game broken telephone – I’ve argued about this before. As stories move from one medium to the next, the context fades away, the caveats are dropped, the uncertainty disappears, and we are left with just a headline.

If you look at the comments on the paper, you’ll see that several scientists have produced well-referenced arguments questioning the methods and the conclusions. The reporter from Chemistry World spent the time to speak with the critics. So the subsequent article presented not only the results of the study, but the controversy over the methods and the specific critiques of the scientific community.

But Chemistry World begat the Reuters and the Times. There, the context was reduced and the criticism of other scientists was gone. Reuters and the Times begat many, many more children, which spread like good little soldiers over the internet. In many cases, now even the context was gone, leaving us with just a headline: Biofuels are worse than gasoline.

This is also an example of what New York Times science reporter Andrew Revkin famously calls the tyranny of the peg (see Matt Nisbet’s discussion in reference to hurricanes and climate change). The peg is the “new study”, in this case a big strong peg, as it is a “new study by Nobel Prize winner”. The tyranny is that instead of a more democratic story about greenhouse gas emissions from biofuel production, including a full reading of the research, we get a linear story just about the latest study.

Think about this. Anyone writing about biofuels and GHG emissions, whether for a science publication, a general interest publication or even a small blog, knows that there have been a number of published studies comparing emissions from biofuel production with emissions from gasoline since those studies also received media attention. So before pounding this particular peg into the ground, there was one obvious question that should have been asked:

How could the N2O from fertilizer offset the GHG reductions from cutting gasoline use, if other studies accounting for GHG from all aspects of biofuel production, including N2O from fertilizer, found that biofuels were, at the worst, a wash with traditional gas? What, then, has this study done to find such a dramatically different result? And how is the scientific community responding?

I’m not advocating for or against biofuels, that's not the point of this post. Nor am I advocating for or against the open review process. I’m asking that we all stop, breathe and think a little bit before reporting new results. Resist the peg.


John Fleck said...

Simon -

With my journalist's hat on, I'm wondering what you think the responsibility of the initiating scientist is in this case. Is it Smith's obligation, in talking to the Reuters reporter, to "saw off the peg" - to explain himself that this is tentative, reviewers disagree - to essentially help along a reporter who is going to be institutionally inclined to misunderstand the scientific process?

Anonymous said...

With my journalists head on. Part of the editing process is likely to be at fault in the way stories become simplified. Leaving aside the lack of understanding of science in the media. People prefer to read or hear stories with a great degree of certainty in them, unless the threat is potentially very large, very scary and very badly defined. Aids in the 1980s was typical of this. The environmental impact of biofuels is certainly badly defined, but doesn't in the public's mind at least, hit the other two buttons. The editing process will tend to build on certainty and reduce uncertainty.
Also, You have to have something that makes the story current or its not news and a celeb always sells (even if they've only won a Nobel Prize and not starred in a Hollywood blockbuster). Most news journalists report stories on one section of society to another separate section of society and often have no link between them, so they don't really care about the nuance of conversations that bind groups together. So you get industrialised news, which is what the wires specialise in. And its what makes publications like Chemistry World and ICIS Chemical Business rather more useful. If less glamourous than the big media.

Simon Donner said...

John - That’s a good question with no right answer. My feeling is there is a shared responsibility. The whole problem would be solved if the reporters set out to write a story not about the “peg” (the new paper) but a story that used the “peg” as a hook to analyze the subject. However, knowing that is not happening, and knowing that research is controversial, the scientists being interviewed should mention the wealth of previous research on the subject, so the reporter knows that more investigation is required on his/her part.

In this case, though, I think the problem is more on the media end, as the last comment mentions. The complicating factor here is that “Nobel Prize winner” is such a strong hook or a big “peg” for a news story. So, in this case, an editor is even more likely to drop all the additional contextual research done by the reporter (no?). That appears to have happened in the Times piece.

Jim Lane said...

Just wanted to make the point that Biofuels Digest covered this angle and didn't just blindly reprint the Reuters piece.

In fact, the "Top Story" on the Biofuels Digest site and email newsletter on 10/2 was titled "Dubious Reuters article on emissions fans anti-biofuels flames". The story went into some about withering criticisms the article received in peer-review, and quoted from the peer reviewers.

Didn't want you to think everyone as taken in by the Chemistry World release, although I agree that Reuters really blew it.

We also contacted Reuters after they misidentified another report as "from the OECD" and never saw a correction published.

Anonymous said...

Good thoughts.
I'm only starting to look at this area, but I've found the following to be useful:

0) International Symposium on Near-Term Solutions for Climate Change
Mitigation in California
March 5-7, 2007
This has a bunch of papers on N2O mitigation, which is an issue with or without biofuels.

1) Chronological Summary of Net Energy Balance Studies Since 1995

2) Ethanol's NEB - Postive or Negative

3) David Strahan, "The Last Oil Shock",; the book (Amazon Canada or UK) is good.

4) Ayres & Warr:
Accounting for Growth: the Role of Physical Work,

It seems pretty clear to me (an old farm boy) that corn was *not* designed as a fuel crop, and is hardly optimal for that, and there is clearly a lot of boondoggle right now ... not an uncommon occurrence.

On the other hand, maybe switchgrass or especially miscanthus, especially with some genetic modification, may actually be pretty good, and need a lot less fertilizer (which comes from natural gas), but for which we don't really have the infrastructure in place, and there is still plenty of research needed. Also, biodiesel may be better for some applications.

Some people think the real (transient) role of corn ethanol is to accelerate distribution infrastructure, and acceptance of flex-fuel vehicles (the incremental cost is maybe $100 right now) so that it's ready for better biofuel crops, and that *is* important, because infrastructure and vehicles do not change overnight.

It is *certainly* the case that North American farmers will grow whatever makes them more money... maybe they will even stop growing tobacco, if miscanthus-ethanol gets really profitable.

Solar-powered tractors already exist, but for sustainable developed-world, but especially for (extreme- energy-intensive) North American agriculture, in the face of Peak Oil 2015 +/- 5 years:

we figure out some biofuel combination that works to replace gas and diesel for medium- and long-distance transport of food; I've seen some ideas for electricity [replaceable batteries for tailer trucks], but I'm not yet convinced.

Cities like New York don't really work, and a whole lot of urban/suburban people go back to approximations of Amish lifestyles, and the huge mid-west farms are chopped back into family farms, and we go back to widely-spread small towns, because even if people can grow the food, they won't be able to ship it very far.
Of course, no matter what, the current practice of shipping low-cost food halfway around the world is going to be over.
Likewise, counting on N. America grain to feed Africa in the long-term seems rather unlikely.

Simon Donner said...

Jim, yes, nice job, I should say that several of the blogs did go the extra mile in investigating this story.

I'd be curious to see how putting a price on carbon, through cap-and-trade, regulation or a tax, will effect the transportation of food. Perhaps it would provide the incentive to do more local distribution. Wishful thinking?

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

It would be a good idea to put taxes on carbon. Transportation causes many greenhouse gases to warm up the ocean. Though it will probably take longer to distribute food around, but it will probably be for the better.