There’s a good good post on Prometheus about a new paper by anthropologist Myanna Lahsen on the culture of climate modelers. The bottom line is the way modelers tend to believe their own models – to substitute them in their own mind for reality. It reminded me of a paper published last year in which B.G. Hunt and T.I. Elliott model climatic conditions in the Yucatan and use them to discuss drought and its implications for the Maya.
Based on proxy records, some researchers have suggested that drought played a role in the collapse of the Classic Maya (it’s debated), and Hunt and Elliott wanted to model the thing.
It’s an interesting paper, with careful discussion of the model’s shortcomings – like, for example, “its inability to reproduce the observed range of SST (sea surface temperature) anomalies associated with El Niño/Southern Oscillation (ENSO) events.” And then they offer this howler. A set of model-generated figures show rainfall anomalies that seem to match up nicely with the proxy records:
These figures confirm that major rainfall deficiences are a systematic, but irregular, characteristic of the Yucatan region.
No, they don’t. They show that the model exhibits systematic but irregular rainfall deficiencies. What they tell us about the rainfall itself is a bit more epistemologically tricky.
This is a bit of a cheap shot, because much of their discussion is generally more subtle, treating the model as a useful tool in exploring the physical system, rather than a direct representation of it. But in the context of Lahsen’s paper, it’s an interesting slip.
Not a cheap shot at all. A very relevant post.
Excerpt:
“modelers had a ‘fortress mentality’. In the words of one such user I interviewed, the model developers had ‘built themselves into a shell into which external ideas do not enter’.”
External ideas like “reality”.
Hi all,
I’m happy to see discussion of my paper.
I want to point persons on this blog to the first comment to my paper on Prometheus, because I think it highlights something some may be inclined to overlook when reading the paper, especially if they are already critical of models.
The person who wrote that first comment described my case study of modelers as “an important example of larger issue both in science and in general perceptions of the world,” noting that “There is a real human tendency for people to confuse their ‘models’ and ‘constructs’ of the world with reality. This is true of economics, psychology and the hard sciences.” He also, correctly, points out that quotes in my paper indicate that modelers as a whole “are very aware of the pitfalls, and to some extent the community is partially self correcting. For example: “It is easy to get a bad name as a modeler, among both theoreticians and observational people, by running experiments and seeing something in the model and publishing the result. And pretending to believe what your model gives – or, even, really believing it! …”
(for the whole comment, see http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/000675myanna_lahsens_late.html#comments )
I write this because my study can be used as a cheap shot against modelers, wherefore it is important to keep the above things in focus as well.
What I hope people walk away with from reading the paper is that “reality” is indeed a difficult thing to know, and that no one – including more empirically oriented climate scientists – have privileged access to it: all renditions of reality are mediated by the methods we use to understand it, and by our conceptual frameworks.
Something that happens a lot in the US politics around climate science and climate change is that one side suggests that they are objective and know reality, by contrast to their opponents. That rhetoric prevails on both sides, and I can see that my paper might be fodder to climate skeptics in that war. I am thus uncomfortable when I see someone, on another blog, say that I argue that modelers have “an unhealthy belief in their models.”
In anthropology, we tend to think that the most robust understanding emerges from considering a plethora of different viewpoints, each of which contributes one part of the whole, and each of which also needs to be scrutinized and understood as rooted in particularity of perspective.
Cheers,
Myanna Lahsen
By the way, I may have invited the interpretation of modelers’ relationship to their models as “unhealthy” because I at one point in the paper write that modelers “do not necessarily have a consistent, ‘healthy skepticism’” in relationship to their models.
What needs to be kept in mind is that the same can be said for the use of models – including theoretical frameworks – in general. I make this point in the context of refuting MacKenzie’s uncertainty trough model for the distribution of uncertainty around scientific productions.
Steve, Myanna –
Thanks for the comments. I hope I didn’t sound as though this was intended as a broad brush criticism of Hunt and Elliott, or of modellers in general. In fact, as I pointed out, the authors did in fact offer thorough and thoughtful discussions of their model’s shortcomings as compared with the available empirical evidence. I think they’re mostly getting it just about right. In that regard, I think Steve’s comment is wrong if he’s suggesting the modellers in this case are igoring “external reality”.
I’m not suggesting that in this case at all. Their insight to many modelers ignoring reality – or maybe “dismissing” reality is more apt – is well known to climate science followers of every persuasion, although maybe “brushed off” more by alarmists and those whose next meal depends on an upcoming climate “crisis”. The fact they realize and face it is admirable.
For example, although both emissions and the annual residual of CO2 in the atmosphere are increasing (the residual being that portion not taken up by the available sinks, and reflected as increasing concentration), the trend of the percentage left in the atmosphere annually is decreasing. That’s the pesky sort of reality modelers can’t (or won’t) duplicate and alarmists are ignoring.
I should add that “at least I haven’t heard of any sort of explanation or assimilation of that trend into their models”.
John, in your “howler” comment in the main post, I’m a little confused (in part because I don’t have the sub to view the paper other than the abstract). You seem to first say that there are proxy records showing the drought pattern, but object when the modelers happily note that their model is able to produce a consistent pattern. If it hadn’t been able to, that would have been cause for wondering whether the model had gotten something really fundamental wrong or if there were errors in the proxy record. In effect, the modelers are saying that they were able to describe an internally consistent climate pattern that agrees with the proxy record, and so confirmed the proxy record (as distinct from the underlying reality of the clinate pattern). What’s wrong with that? As you know, proxy records require interpretation, and maybe there was some question in that regard. If not, perhaps it would have been better for them to say that the agreement between the two tends to confirm both. But in any case I don’t think it was quite fair to call it a howler.
On the subject of Myanna’s study, I’m glad to see her state that the effect isn’t really specific to climate modelers or even modelers of any kind. There are examples of “pet theories” throughout the science biz, many of which have nothing to do with models. What is arguably different about climate models is that there are far more details to get lost in than is the case with most other fields. OTOH, what I think saves climate modeling from losing its way (even if from time to time some individual modelers do) is the existence of so many competing models. A given modeler might have a less than objective view of her/his own model, but I’ll bet that doesn’t extend to being uncritical of the models of others. If anything, human nature being what it is, I would expect the opposite.
The models don’t include variability in sinks, Steve? Well, that *does* seem like a huge defect. Thank goodness someone spotted it. With that settled, would you mind having a look at the cloud problem?
LOL – I haven’t settled anything, and I can’t even claim credit for “spotting” what appears to be a major biospheric response. I disagree with John in that I don’t think modelers are anywhere close to solving the question of what will happen under increasing CO2, particularly regionally. You bring up a good point about clouds as well.
A lot of modelers seem to think they can just vote on reality, and the fact that they mistake model outputs as data is disheartening, to say the least. For some of them it’s as if the computer gaming generation has found a gravy train. Kind of tragic too, when we consider the primary response of the biosphere to increased CO2 is to grow more flora.
The lapse rate in the troposphere is adiabatic, and since gravity and the mass of the atmosphere aren’t changing (significantly) I’m not sure how these nightmare scenarios are justified, especially since about a billion people on Earth are on the verge of starvation, and increasing CO2 increases flora, aka food.
I just think we need to increase the amount of research on climate change by at least an order of magnitude, and ensure the researchers stay “tightly coupled” to reality. The perversion of the scientific method highlighted by the MBH Hockey Stick and it’s bandwagon of supporters needs to be fixed.
Steve B –
The reason I thought it was a howler is that the modellers in the sentence I quoted are saying that their models have confirmed a specific physical reality suggested by the proxy data. They’re not saying that their models are consistent with the proxy data. But to repeat – it’s a cherry-picked quote, because much of the paper involves a more sophisticated discussion of the model’s relative consistencies and inconsistencies with various physical records both in the Yucatan and elsewhere.
Steve H. –
I can’t let you get away with your continued shots at MBH. It was a significant piece of cutting edge research. A lot of good science has been done on the question since, some of which confirms it in important details, some of which disagrees with it in important details. That’s how science works. Every result is contingent. To call it a “perversion of the scientific method” is a misrepresentation.
John,
Do you not believe that random numbers put into the MBH procedure result in a hockeystick shape?
The question is not “how close is the hockeystick to fact”. That’s irrelevent to the larger question. The larger question is “How can such a fraudulent procedure be labeled as science”? No matter how much people repeat it as truth, a lie is still a lie.
Steve H –
I’m not a statistical climatologist, so I’ll defer to Hans von Storch on your question. In GRL last October, he concluded that red noise put into the MBH procedure did not automatically result in a statistically significant hockey stick shape, as M&M have claimed.
Two points worth making on von Storch. First, he is clearly no friend of the hockey stick. Second, he is the author of one of the standard texts on statistical climatology. This doesn’t mean von Storch is right. Steve McIntyre has a counter-argument. My purpose here is merely to point out that it is not as drop dead obvious as you seem to think it is that MBH is a “fraudulent procedure.”
John,
There are different levels of “statistically significant”. One can always call up a more extreme definition of “significant”.
The fact is, it *usually*, and on average, does result in a hockeystick shape. If the average is one standard deviation off and the definition of “significant” is two standard deviations, then of course there is not a “statistically significant” result.
Here is that actual Science paper of Von Storch et al:
http://tinyurl.com/bjxmn
And, here is what your Von Storch said in Der Spiegel, a German magazine. (actually the English translation)
“We were able to show in a publication in Science that this [hockey stick] graph contains assumptions that are not permissible. Methodologically it is wrong: Rubbish (or Junk[1]).”
*[1] From the German phrase Quatsch
Von Storch goes on to say:
“The Mann graph [i.e., the hockey stick of IPCC TAR] indicates that it was never warmer during the last ten thousand years than it is today. … In recent years it [the hockey stick] has been elevated to the status of truth by the UN appointed science body, the Intergovernmental Panel on Climate Change (IPCC). This handicapped all that research which strives to make a realistic distinction between human influences and climate and natural variability.”
Note that within the last 10,000 years the Sahara has been a verdant plain, and within the last 1,000 years Europeans were farming Greenland.
Steve H –
As I said, and as you point out, von Storch is no friend of the hockey stick. I’ve written in the past about the Science paper you cite. As I’ve said before, I think it’s a significant result, raising questions about whether the MBH methodology would understate centennial-scale variability. Subsequent to that, some methodological questions have been raised about von Storch’s Science paper, some of which he implicitly acknowledges in the GRL paper of last October to which I’m referring. That’s all a robust methodological debate, which is healthy.
You allege fraud. You used the words “fraudulent” and “lie.” In support of that claim, you cited what von Storch calls the “artificial hockey stick” (AHS) effect as evidence that the methodology is “fraudulent” and a “lie.” Von Storch, in his GRL paper (using the same methodology he used in the Science paper which you approvingly cite above) concluded that “the AHS does not have a significant impact but leads
only to very minor deviations.” He still doesn’t like the methodology, as he reiterates in the GRL paper. But that’s an argument among scientists about the proper methodological approach to a cutting edge question. That kind of thing happens all the time. It’s ordinary science. It’s a far cry from fraud.
John –
I agree with you that the hockestick itself is not fraud. What is fraud is its continued support after M&M, Von Storch, etc. have debunked it – and I’m not talking about the typical layperson who believes it – I’m talking about climate scientists who still say it’s okay and convince the typical layperson.
Unless, of course, there are statisticians who say the hockeystick is okay, in which case the question is still open…
Steve H –
There are a whole bunch of climatological statisticians who say the hockey sticks (plural – there are any number at this point) have validity. A bunch of different proxy datasets, and a bunch of different methodologies all point in the same general direction: warming in the 20th century that is different than anything that’s happened on millennial time scales. There’s not just one hockey stick. It’s worth sorting out the arguments over MBH, but we could throw out MBH completely and we’d still have pretty much the same picture at this point. The various sticks disagree in their details, especially about the magnitude of past variability (the point von Storch has harped on). That’s where the robust and interesting scientific debate is going on today.
There are honest and reasonable climate statisticians who say MBH is OK as well. This is the reason for my frustration with your use of “fraud” and “lie,” because those words suggest intentional deception where what we really have is a reasonable disagreement over the tough scientific questions raised by the MBH method itself.
But that’s not the most important point to me. The most important way science moves forward is finding other data sets and other methodologies for looking at a question. And on that far more important score, the hockey sticks have held up fine.
Finding more data and better ways to analyze it is indeed the most important thing.
However, you missed my point. My point is not whether or not the granddaddy hockeystick – MBH98 – is reasonably close to reality.
It’s because the method is invalid and reduces past variability, which (coincidentally?) make things look worse than they really are.
http://www.sciencemag.org/cgi/content/full/306/5696/621/F1
So, it’s not how close it was, it’s that the method (whether or not Mann et al knew it at the time) was faulty. That was all that Mann et al were guilty of at the time – of being wrong. Since then has been the deception of continuing to use it because it can be thrown around amongst a host of other bent up hockeysticks that more closely represent what really was, and are themselves more valid. MBH98 is not *valid*. We will see what AR4 uses.
By the way, I wonder when we’re going to get a warmer year than 1998? The nose of all hockeysticks seems to be rubbing off now…
Steve, two out of three (NCDC and GISS) say 2005 *is* warmer than 1998. A close thing statistically, of course, but when you take into account the fact that global temps are puffed up by strong El Ninos, 2005 is the clear winner (and by more than a nose). BTW, 1998 is such an outlier that it doesn’t pull the current trend down much even if the El Nino effect is ignored.
Pingback: jfleck at inkstain » Blog Archive » 2005 Temperatures
Steve –
The real point is, when the alarmists are saying that global warming is accelerating, we’re even having this discussion. Obviously GISS has an agenda – cherry picking the temperature gains of “the last 30 years”. What’s the trend of the 30 years before that? Yep, decreasing. Was CO2 decreasing then? I don’t think so…
There’s no doubt temperatures are increasing, but the real question is why? One big change of the last couple of decades is albedo of the Arctic, from particulate pollution of the wildly increasing dirty emissions of China. Can’t say I blame them, but let’s not go off calling CO2 the bad guy when we don’t know how much of an effect it’s actually had (what about convection?). CO2 gain is probably the single biggest thing increasing food availability in the third world – and God knows they need it.
If every piece of evidence in favor of anthropogenic global warming can be dismissed as a conspiracy by some sub-set of those thousands of scientist-conspirators, then there’s not much point to the discussion, is there? Fortunately most people are a bit more willing than you to examine the evidence. It is sort of amusing that you first ignore the obvious explanation for the temp dip circa 1940-1970 (aerosols), and then blame them for the being a major factor in the current warming (which they aren’t, local effects from ground deposition in the Arctic notwithstanding). But do keep on contradicting yourself like that.
All aerosols are not created equal. That must be on the other side of your Tower of Babel.
http://www.agu.org/pubs/crossref/2005…/2004JD005432.shtml
Try to keep an open mind Steve. Don’t be so obviously prejudiced.
Bad link.
Try this referring link:
http://tinyurl.com/blue7
I actually got it from John on this page:
https://www.inkstain.net/fleck/?p=1308
Steve H –
In the interest of pushing this discussion in a useful direction, rather than just having antagonists snarking at one another, could you elaborate on what you think the importance of the Park et al. paper is, since you’re the one who’s been bringing black carbon albedo issues into the discussion?
What does this paper and the other literature suggest is the size of the black carbon forcing, and how does it relate in size to the other more familiar forcings typically discussed?
Thanks.
From the Park paper referenced here:
http://tinyurl.com/blue7
“Previously calculated decreases in visible snow
albedo due to BC are 0.8–4.5% with 10 ppbw BC and
1.9–9.5% with 30 ppbw, depending on the BC mixing state
with snow and the age of the snow [Hansen and Nazarenko,
2004]. Our simulated BC concentration in Arctic snow then
implies an associated decrease in snow albedo of 3.1 ± 2.5%
based on linear interpolation from the ranges above.”
Briefly,
Incoming radiation is about 342 w/m^2 (watts per square meter) and one fairly accepted number for increased heat retention due to greenhouse effect is about 2.5 w/m^2,
http://www.faqs.org/faqs/sci/climate-change/basics/
In the polar regions average insolation is a little over half the average, so we can say (conservatively) 170 w/m^2. 3% of that is 5 w/m^2 – but it could be anywhere from a little less than one to almost 10. So, it could be from about 1/4 as much to 4 times more.
That’s a typical uncertainty with climate science – and that uncertainty is why we shouldn’t start wasting money on fixes yet, but definitely should spend money on research because other albedo changes and convection play major, yet unknown, roles.
Steve –
Thanks for this. It looks from these numbers as though the albeda change from black carbon is substantially smaller than the albedo change (30 percent) found by Sturm et al. (JGR, doi:10.1029/2005JG000013) resulting from warming-induced increased shrub abundance.
On the convection issue, are you arguing that the current models do not properly take into account the convection you and Roger were discussing over on his blog?
Steve –
First, thanks for raising this moist convection issue, which is not one I’ve spent much time on. In fact, it turns out there’s a guy here in New Mexico who’s done a great deal of research in this area – Ken Minschwaner down at New Mexico Tech. Ken and colleagues, including Andrew Dessler at Texas A&M, have been using satellite data of the troposphere over the tropics to see whether the models accurately capture what’s going on. Their conclusion, in a paper currently in press with the Journal of Climate, is that the models being used for the fourth assessment “are simulating with reasonable accuracy the observed behavior of water vapor in the tropic upper troposphere.”
If albedo changes from black carbon are 1/30th as large as from plant growth increase, then there doesn’t seem to be much left for the greenhouse effect…
It will be interesting to see what the paper actually says when it comes out. One thing that seems to get lost when the bandwagon riders look at a paper are the words “may” or “could” – usually in the first paragraph…
Interestingly, in terms of model outputs, there seems to be a new paradigm that says if we run all the models and take the mean, the mean must be close. It’s a re-invention of the concept that model outputs are data, which of course they’re not. A lot of modelers don’t seem to grasp that concept though.
The plant growth increase is a positive feedback resulting from warming, according to Sturm. Since the potential black carbon effect looks small, and the shrub expansion follows warming, it looks like we need another mechanism. Greenhouse hypothesis seems to fit nicely. To say “there doesn’t seem to be much left for the greenhouse effect” ignores the question of what’s caused the warming that’s leading to the expansion of the shrubby vegetation in the first place. Other suggestions?
On the question of the model outputs and data, the whole point of the Minschwaner paper is to compare model outputs with empirical data. Minschaner et al. aren’t taking all the model outputs and taking the mean and saying it’s data. They’re taking the model outputs and comparing them to *actual data* to see if the models are getting it right. They conclude that the models are getting the convection right.
As I read about it, it seems the convection issue you’ve been discussing is a well understood phenomenon that the modellers have taken into account. So I repeat the question I posed earlier. Is your purpose in raising the convection issue to suggest that you think the models are getting this wrong?
It’s worth pointing out the answer to my first question posed above, as given by Hansen and Nazarenko, who Park cite as the authority for their calculation of albedo on snow. They conclude that “The soot effect on snow albedo may be responsible for a quarter of observed global warming.” The other three quarters? “(A)nthropogenic greenhouse gases have been the main cause of recent global warming and will be the predominant climate forcing in the future.”
I misread your earlier post, but still there’s not much room for the “greenhouse effect”. According to your numbers, if warming from black carbon is only 30% as large as from plant growth changes, and it’s 25% of the total, then that means plant growth changes are 25%/30% or about 85% of the total. That means, by your numbers, that we’re already over 100%.
The caveat, of course, is that the uncertainty is so large none of these numbers mean anything in terms of reality.
One comment though – the increase in plant growth is not due to warming so much as to an increase in plant food, aka CO2. I don’t think we can attribute the increased plant growth in the Sahel to “warming”.
Now, extrapolate what is happening happen up north. More forests and arable land, less swamps… anywhere else that would be called “reclamation”.
P.S. Polar bears did fine when Europeans were farming Greenland.
P.S. #2 – Do you have the actual paper by Minschwaner et al? If not, maybe we should wait and see what it actually says *after* it’s published so we can incorporate its uncertainties as well.
They’re not my numbers. They’re Hansen and Nazarenko’s numbers, which you yourself brought up when you quoted them via Park – but quoted them incompletely. You’re conflating albedo changes (the percentages you’re using in your most recent comment) with climate forcings, and then ignoring what Hansen and Nazarenko have to say on the very point you’re trying to discuss – the relative importance of black carbon albedo changes and greenhouse forcings in climate change. Hansen and Nazarenko say greenhouse gas forcings are significantly larger than the albedo changes from black carbon.
The irony here is that Hansen and Nazarenko support your underlying argument – that black carbon is an important climate forcing mechanism that has been inadequately addressed. But in reaching beyond that point to argue that “there doesn’t seem to be much left for the greenhouse effect,” you’ve cherry-picked their data to support an argument at odds with one of their central conclusions.
This is very much like what you did with the Hansen quote from Science magazine: a partial quotation, taken out of its original context, that seemed to you to be saying something very different than a broad reading of the entire context showed.
As for the Minschwaner paper, yes I do have it. But let’s set it aside, and I’ll reiterate the question you haven’t answered. Is it your position that the current models do not properly handle the moist convection issue you raised? If so, could you point me to some literature to support that position?
On warming-vs.-CO2 as a mechanism for the spread of shrubby growth into arctic tundra (and we are talking here about shrubs in the tundra, not the Sahel), Sturm (Sturm, M., T. Douglas, C. Racine, and G. E. Liston (2005), Changing snow and shrub conditions affect albedo with global implications, J. Geophys. Res., 110, G01004, doi:10.1029/2005JG000013) suggests warming is “the most likely cause.” It seems a reasonable argument, and he’s citing what looks like a broad literature to support it, but I haven’t begun to read all the literature he cites, so I could be missing something here. Perhaps you have some other citations on the expansion of shrubby ecosystems in arctic tundra you can share?
You obviously missed two things in that post. Maybe because you’re sick. One was the tongue-in-cheekness of the statement “doesn’t seem to be much left for the greenhouse effect” and was intended to point out the futility of any quantification, at this time, of climate change. We just don’t know enough. Nobody does. You might review your math above – you have attributed the same ~75% to both vegetation enhancement (Sturm) and the “greenhouse effect”.
The other thing you missed was actually a more direct comment to the same point, and is the most important thing about the pseudo-debate of climate change:
“The caveat, of course, is that the uncertainty is so large none of these numbers mean anything in terms of reality.”
And that’s what I’ll leave you with.
Your suggestion that I’m attributing the same 75 percent to two different things compares apples and oranges. One is Sturm’s measurment of albedo change as compared to the albedo change of black carbon at a particular spot that has one or the other. The second is Hansen’s attempt to quantify the integrated effect of black carbon globally to greenhouse gases globally. Two completely different things, not “the same 75 percent.”
Thanks again for raising the issues of black carbon albedo and moist convection. I know a great deal more about the literature on both as a result of our discussions.