Winds of Change.NET: Liberty. Discovery. Humanity. Victory.

Formal Affiliations
  • Anti-Idiotarian Manifesto
  • Euston Democratic Progressive Manifesto
  • Real Democracy for Iran!
  • Support Denamrk
  • Million Voices for Darfur
  • milblogs
Syndication
 Subscribe in a reader

Wow. Just Wow.

| 198 Comments

I've been unhappy with the quality of data released to support AGW, and so was unwilling to jump onto the bandwagon - while supporting things like energy independence. And I've been worried that core data - which keeps somehow being unreproducable or unavailable - needs to be rigorously reviewed before we make critical policy decisions.

But I never expected outright fraud.
From: Phil Jones To: ray bradley ,mann@xxxxx.xxx, mhughes@xxxx.xxx Subject: Diagram for WMO Statement Date: Tue, 16 Nov 1999 13:31:15 +0000 Cc: k.briffa@xxx.xx.xx,t.osborn@xxxx.xxx

Dear Ray, Mike and Malcolm,

Once Tim's got a diagram here we'll send that either later today or first thing tomorrow.

I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to hide the decline. Mike's series got the annual land and marine values while the other two got April-Sept for NH land N of 20N. The latter two are real for 1999, while the estimate for 1999 for NH combined is +0.44C wrt 61-90. The Global estimate for 1999 with data through Oct is +0.35C cf. 0.57 for 1998.

Thanks for the comments, Ray.

Cheers Phil

Prof. Phil Jones Climatic Research Unit Telephone +44 (0) xxxxx School of Environmental Sciences Fax +44 (0) xxxx University of East Anglia Norwich Email p.jones@xxxx.xxx NR4 7TJ UK
This is mindblowing.

What we have is a body of research supported by hundred of millions in research grands based on a premise promulgated and legitimized by a guy who is well on his way to making a billion from claiming to manage it.

And I thought it was bad for hucksters to 'create' issues that they could then sell patent medicine to cure.

Update: There's a great precis of the "juicy" emails over at the Bishop Hill blog. Go read it, and then decide a bit more how you feel about this...
-

198 Comments

Investigate magazine is moving fast on this:

TGIF asked Jones about the controversial email discussing hiding “the decline”, and Jones explained he was not trying to mislead. “No, that’s completely wrong. In the sense that they’re talking about two different things here. They’re talking about the instrumental data which is unaltered – but they’re talking about proxy data going further back in time, a thousand years, and it’s just about how you add on the last few years, because when you get proxy data you sample things like tree rings and ice cores, and they don’t always have the last few years. So one way is to add on the instrumental data for the last few years.” Jones told TGIF he had no idea what me meant by using the words “hide the decline”. “That was an email from ten years ago. Can you remember the exact context of what you wrote ten years ago?”
The other emails are described by skeptic commentators
as “explosive”, one talks of stacking the peer-review process to prevent qualified skeptical scientists from getting their research papers considered

I've been worried that core data - which keeps somehow being unreproducable or unavailable - needs to be rigorously reviewed before we make critical policy decisions.

The problem is that the core data set that was in the custody of Phil Jones and his colleagues has been lost or destroyed. According to one version of the story, it was on an old drive that was discarded.

Later Jones denied that it was lost, saying that they could reconstruct the data from the reports they wrote for the US Department of Energy. Why they would have to reconstruct it if it wasn't lost, well ...

Jones previously refused to release data to people who were going to "try to find something wrong with it."

That these people have departed completely from science is too obvious to need stating, which is good, because politics forbids us to state it.

This looks pretty damning, at least in terms of manipulation of the review and publication process. However, it might be a good time to keep the '48 hour rule' in mind until those who have some knowledge of the domain, the personae involved, and who can do some forensics on files and e-mails have weighed in.

I'll just note that given the time interval in question, the e-mails can only be a small fraction of those sent and received. So they must be cherry picked in some sense - not saying whether good or bad. The original hacker/leaker called this a 'random selection' so there may well be more than will / could come out. Someone may be learning from Breitbart's tactics in dribbling out his most damning evidence.

BTW, what has science come to, when scientists are filing Freedom of Information requests against other scientists, which are then evaluated by lawyers?

Keep in mind the recipients of that email were Michael Mann, Raymond Bradley, and Malcolm Hughes. Of the famous hockey stick, heavy hitters in the global climate game. This is NOT an interoffice email of the CRU- Bradley works at Univ of Mass Amhurst dept of geoscience, Malcolm Hughes is a tree ring and ice core expert from the University of Arizona, and Mann of course had a huge impact on the UN IPCC report while working at the Univ of Virginia.

Either this is the worlds most startling coincidence (given the hockey stick controversy) or there damn well IS a cabal of sorts.

Moreover- that's may not even be the most shocking email. Word is there is talk of stacking the scientific journals to intentionally keep out skeptical papers. THAT would be immensely shocking, as the argument goes that skeptics have no credibility because they aren't published. But if they aren't published because they are being blackballed, we have a serious, serious problem.

Is this the premise? ["There is a positive correlation between the presence of “greenhouse gasses” and the absorption/retention of heat in the atmosphere such that an increase in greenhouse gas will cause an increase in environmental temperature. Carbon Dioxide (CO2) is one of these greenhouse gasses."]

If not, which premise are you speaking of? Whose premise? Prof. Phil Jones? How is that person you are referring to on his way to making a billion dollars by claiming to manage it? What is it? How do hundreds of millions of research grants translate into a billion dollars for one person? Are you suggesting there is one person who gets all the credit, or blame, for all of the global warming discussion?

"The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t. The CERES data published in the August BAMS 09 supplement on 2008 shows there should be even more warming: but the data are surely wrong. Our observing system is inadequate"

This is crazy. The data doesn't match the models therefore something must be wrong with the data? What kind of science is that?

All I can say is I am concerned with the veracity of these emails if only because they fit too perfectly into the stereotype of smugness and total self-assurance the AGW leadership display at all times. As AL notes, it also lines up with the odd way analysis gets fed out, often without raw data. I noticed this with the satellite data that never matched up with ground data, and then magically somebody came up with some algorithm that lined them up, and only in retrospect was absolutely proper and necessary to apply. Too convenient... and its not even that convenient.

The above quote was supposedly written by Kevin E Trenberth. Lead author of the 2007 IPCC assessment.

Roland, sometimes I wonder if you're being deliberately obtuse.

"The claim" is that AGW (Anthropogenic Global Warming) exists - that the world's temperatures are rising AND that that rise is predominantly caused by human action, primarily CO2 emissions.

Your 'claim' is laughable on its face as a prescription for policy - AGW isn't.

The guy who will become the world's first carbon billionaire is Al Gore, who singlehandedly legitimized AGW, continues to spend most of his time publicizing it and arguing for it, and also is heavily invested in companies that will profit from steps to deal with AGW.

next?

Marc

The e-mail traffic has been made searchable - at least for now. Read and make up your own mind.

This is mind-blowing too:

http://i45.tinypic.com/iwq8a1.jpg

You can download the archive for yourself here

I've scanned the file and found no malware, but check for yourself. Busy reading this weekend!!

Marc

A.L.

Sorry, I'm not trying to be dense. I make no claims regarding global warming. I understand it to be a scientific debate and my perception is (but I have no way of judging) that a consensus has emerged in the scientific community that human activity is indeed influencing global warming. What the ultimate effect will be of this, how important it is to reverse the warming trend, and how this could best be accomplished I don't know enough about to have an informed opinion.

I'm not sure what Al Gore has to do with these emails. He has been advocating global warming as a concern to be taken very seriously for thirty years or more. It also appears he has done well for himself since 2000. Good for him.

Net worth of $1 billion is kind of silly. There seems to be some speculation that his net worth is ~$100M, but frankly I'm pretty skeptical of that without seeing his financial statements. That sounds like an unreaslistic sum to have accumulated in nine years. As to his investments in green technology--it's a free world. You and I and Warren Buffet can freely invest in green technology. It's a growth industry, and innovation is good. If these businesses can be succesful, great. If they are successful, it will help America stay on the forefront of technological development. That sounds all good no matter how the AGW evidence ultimately shakes out.

So, do I understand correctly then that the point of your original post is that AGW is a conspiracy of Al Gore and fraudulent scientists cooked up in order to enable Al Gore to get rich?

So, do I understand correctly then that the point of your original post is that AGW is a conspiracy of Al Gore and fraudulent scientists cooked up in order to enable Al Gore to get rich?

Well, no, the topic appears to be putative fraud, collusion and manipulation of the scientific review and publication process on the part of many well-known AGW proponents. No one else seems to be having trouble figuring it out.

First blush with these emails- these guys are obsessed with what the skeptics are saying and doing.

McIntyre (of Climate Audit) is mentioned in 109 different emails.

The word 'skeptic' in 105.

Mind you, thats in a little over 1000 emails total.

mark: Agreed about the obsession, but be careful with using 1000 as the denominator: Whoever "randomly"(!?) selected these messages might have done so by grepping with just the type of key phrase you are using.

For those who've grabbed the document set (Marc et alia), it's worth a look at the one called "HARRY_READ_ME.txt". It's not your usual README, but rather a very long and involved narrative of an analyst's attempt to revive and use (perhaps over a period of years, 2006-9) both datasets and programs used in earlier studies. Lots of console clips interspersed with commentary. I surmise that "Harry" could well be the "Mr. Ian (Harry) Harris" in the CRU directory listing his work areas as "Dendroclimatology, climate scenario development, data manipulation and visualisation, programming" which certainly seems to fit.

For someone with the expertise and patience to chug through it, this file appears to be pretty much a gloss on the original data, likely many of the manipulation and analysis programs contained in the cru-code folder (much of it is apparently recent generation FORTRAN), as well as the various alterations to both that were apparently necessary to replicate earlier results.

That sounds like an unreaslistic sum to have accumulated in nine years.

Not when suckers like Mike Schmidt (Google) and Steve Jobs (Apple) make you a director/senior advisor and give you shares. Those guys are smart businessmen, but I suspect their motives were a mix of misguided idealism and pragmatic politics. In any case, Gore hasn't done anything worthy of the money, IMHO.

Al Gore, of course, has recently discovered the real cause of global warming: stellar fusion at the earth's core!

Since we can't have more than a few minutes left before we become a companion star of Sol, I'd like everybody to know that I feel really bad for making fun of Al Gore so much.

I don't get the impression that he is having any trouble figuring it out, just that he's trying to misdirect the conversation. I could be wrong, of course, particularly since I don't know his posting background and thus don't want to judge such a thing on the basis of a couple of comments on one thread.

For context, here's Real Climate's take on the emails

It seems worth pointing out, as RC does, that these were, in fact, stolen emails, and I suspect that had these been stolen emails from, say, an energy company, and liberals were jumping up and down on an off-the-cuff, out of context remark as proof of a vast conspiracy, Armed Liberal and the usual commenters on this site would be tut-tutting about how unethical it is to comment on illegally obtained communications.

That said, the idea that Al Gore "singlehandedly legitimized AGW" is ludicrous, as is the implication that he's somehow manufacturing a crisis to get rich. It's simply a conspiracy theory of the highest order, and it's frankly a sign of how vastly out of whack AL has gotten that he'd seriously make that argument.

Chris -

When the "Pentagon Papers" (top secret documents rather more momentous than anybody's email) were stolen, they were published on the front page of the NYT. The NYT argued that they had a First Amendment right to publish material that was relevant to government policy, even if the material was stolen.

While I would not agree with the NYT's so-called reasoning, the spirit of it makes far more sense here. This research has been funded by at least two governments, and may affect the policies of dozens of countries. Unless democracy is a complete farce, people have some right to know about it.

While the work of the CRU, et al., is not classified, they have acted as if it is. They have stonewalled people asking for rightful access to their data, and if these emails are not forgeries, they have sought to undermine the UK Freedom of Information Act - perhaps even conspired with government officials to do so. And they've played the perfect McCarthyites with their critics, who include no small number of people.

Even if they have broken no law, they've been a bunch of bastards about this and they deserve to wear a big stinky albatross.

Besides, the word around the campfire is that Russian hackers stole the data, maybe even with the connivance of the Russian government. When the Russians steal stuff, they do it for world peace, right?

Regarding Real Climate's take: They have a point about informality being taken out of context - scientists are only human beings, after all. But they go on to protest far too much. It's one thing to claim that "trick" does not imply deception, but it's silly to claim that "hide" means anything except hide.

If nobody has anything to hide, then let's have the lid off everything, please.

Chris, looking at the emails in a fast scan (as I have), Real Climate's claim that these are simply scientists informally and possibly rudely doesn't stand up. There is clear data-fudging, clear manipulations of process, and other things which make a bit of a travesty of the scientific process.

As to my supposed tetchiness if these had been oil company documents - based on what, exactly, do you make that claim?

And finally, my earlier discomfort with AGW was based on the unwillingness of the proponents to open their kimonos and show underlying data and models, demanding instead that we accept their claims as validated by like-minded people.

That's not how science is done, and especially that's not how science that's supposed to drive global policy is done.

After the publication of these emails, my hope is that AGW proponents won't be able to do that any more, and that we'll have the full and frank discussion we should have had a decade ago.

So to put you on the spot, Chris - do you support that or not?

Marc

This is going to be... interesting.

I'm still going to take a wait and see attitude, because if I wasn't qualified to judge the scientific data on global warming last week (and I wasn't), the e-mails would have to be really, really egregious for me to be able to judge them today.

I'm also not a conspiracist by nature.

I do have a sense for how people in large organizations (in my case, a large corporation) act when they believe in something and have a huge mess of data that is incomprehensible in raw format to the people they want to convince. There absolutely are strategy sessions that amount to, "How do we present this?" and, "How do we get this through?" There is a constant ethical dance, trying to figure out where the line is and how to get as close as possible to it without crossing over it.

This, I understand. It happens in every industry there is. One expects scientists to be a little more pure and a little less susceptible to this on first blush, until one remembers that not only are scientists (some of them) driven by ego in the same way corporations are driven by greed, but also... scientists can be driven by greed based on the award of research grants and funding.

So as I said, I'll be waiting and seeing and following the story carefully.

From the Bishop Hill blog,

Santer complaining about FoI requests from McIntyre. Says he expects support of Lawrence Livermore Lab management. Jones says that once support staff at CRU realised the kind of people the scientists were dealing with they became very supportive. Says the VC [vice chancellor] knows what is going on (in one case).(1228330629)

I wonder if "lab management" might refer to Steven Chu, Obama's current Secretary of Energy?

Chris, looking at the emails in a fast scan (as I have), Real Climate's claim that these are simply scientists informally and possibly rudely doesn't stand up. There is clear data-fudging, clear manipulations of process, and other things which make a bit of a travesty of the scientific process.

This is false, simply put. I have a doctorate, I've worked in scientific academia, and these emails are very much in line with how people talk and deal with data. You, on the other hand, have no scientific background to make these claims that I'm aware of, and the fact that you're confusing a scientific issue with your existing antipathy towards Al Gore even further discredits any conclusion you're making.

As to my supposed tetchiness if these had been oil company documents - based on what, exactly, do you make that claim?

Why, your long-standing disinclination to mount any serious critique of conservative interests, of course, which is more than evidenced by the subjects of your posts on this site for the past half-decade.

And finally, my earlier discomfort with AGW was based on the unwillingness of the proponents to open their kimonos and show underlying data and models, demanding instead that we accept their claims as validated by like-minded people.

And this is, in and of itself, a spurious claim promoted by die-hard AGW skeptics - that the data has never really been shown, that real skepticism has never really been paid to the science behind AGW.

In fact, there are tons of cases where skeptics have been positive they found some massive flaw in the data, only to be proven wrong again and again. There was the hockey stick controversy , since discredited by a National Research Council report, among other reviews. There was Watt's insistence that he'd found systemic errors in the surface station network, since rebutted by the NOAA . The entirety of Real Climate is basically one long list of them patiently and successfully rebutting the claims of skeptics who insist that the science is flawed, only to be swatted down again and again.

Are there cases where valuable data has been lost? Absolutely - that's not at all uncommon in science. (We taped over the freaking MOON LANDING , remember?) But if you actually follow the back and forth of climate debate - I have, and I'm very skeptical you've done the same outside of skimming through conservative talking points on the subject, based on your arguments here - then it's hard to come to the conclusion that there's somehow been some withholding of vital information in the AGW debate. Even basic reading of AGW literature makes it clear that there are multiple, independent sources of data suggesting AGW, from ice cores to tree rings, from satellite readings to ocean temp buoys. The idea that finding some kind of smoking gun with one of these areas of research will somehow undermine all of AGW theory is misinformed; the idea that the underlying data supporting AGW hasn't been properly examined is, at best, ignorant, and at worst willfully foolish.

Marcus V. (#23),

There absolutely are strategy sessions that amount to, "How do we present this?" and, "How do we get this through?" There is a constant ethical dance, trying to figure out where the line is and how to get as close as possible to it without crossing over it.

Yeah, but I'll take Feynman's approach any day--otherwise let's not dignify the process and the people with the words "science" and "scientist".

I have a doctorate...

Oh, wow. Hat's off, gentlemen, a genius. Any of you other ignorant yokels here have a doctorate?(sheepishly raises hand).

@Chris (#25) you can try but you can't white wash CRU's consistent evasiveness in providing data. You say that accidentally losing data does happen sometimes, and it does, but there is a pattern here. The CRU leaks show at least 5 examples of discussion of ways to intentionally avoid releasing data, avoid compliance with Britain's FOIA, and specifically mention the technique of deleting data rather than releasing it. Given the context, that alone is incredibly damning of these folks scientific credibility.

Also, I hope to hell that the rest of the scientific community shouts down the nonsense sentiment that "this is how scientists everywhere operate". That is an absolute falsehood and a horrible insult on a great many professional scientists in the world.

For my part, I can't understand why these guys weren't publishing everything to begin with. Internal emails and personal sniping are one thing, but their code and data are presumably funded by the UK government and should be public.

After all, extraordinary claims require extraordinary proof, and the AGW guys are making claims that are being used to redirect trillions of dollars worth of human activity all over the world. One would think that pissy academic nonsense should be put aside while they make their case in front of the world.

Now, it sure looks like an honest-to-Gaia conspiracy by True Believers to shove An Agenda down the world's throat based on bogosity.

Chris:
The entirety of Real Climate is basically one long list of them patiently and successfully rebutting the claims of skeptics who insist that the science is flawed, only to be swatted down again and again.

Yes, that patient cherry-picking process is described in this e-mail.

Anyway, I wanted you guys to know that you're free to use RC in any way you think would be helpful. Gavin and I are going to be careful about
what comments we screen through, and we'll be very careful to answer any
questions that come up to any extent we can. On the other hand, you
might want to visit the thread and post replies yourself. We can hold
comments up in the queue and contact you about whether or not you think
they should be screened through or not, and if so, any comments you'd
like us to include.
You're also welcome to do a followup guest post, etc. think of RC as a resource that is at your disposal to combat any disinformation put
forward by the McIntyres of the world. Just let us know. We'll use our
best discretion to make sure the skeptics dont'get to use the RC
comments as a megaphone...

It's too bad they don't have that level of control over everything, but not for lack of trying.

PS Re CR, I do not know the best way to handle the specifics of the editoring. Hans von Storch is partly to blame -- he encourages the
publication of crap science 'in order to stimulate debate'. One approach
is to go direct to the publishers and point out the fact that their
journal is perceived as being a medium for disseminating misinformation
under the guise of refereed work. I use the word 'perceived' here, since
whether it is true or not is not what the publishers care about -- it is
how the journal is seen by the community that counts.
... Mike's idea to get editorial board members to resign will probably not work -- must get rid of von Storch too, otherwise holes will eventually
fill up with people like Legates, Balling, Lindzen, Michaels, Singer,
etc.

Oh, wow. Hat's off, gentlemen, a genius. Any of you other ignorant yokels here have a doctorate?(sheepishly raises hand).

That's a bullshit argument, chuck. I bring up my experience in science as a way to invalidate AL's argument that this is bad science. If you have similar experience that leads you to believe otherwise, have at it, but don't dodge the point by playing this into a nonsense "elitists vs. everyone else" argument.

Also, I hope to hell that the rest of the scientific community shouts down the nonsense sentiment that "this is how scientists everywhere operate". That is an absolute falsehood and a horrible insult on a great many professional scientists in the world.

And your basis for saying this is, Robin...? Do you have any relevant experience in conducting research and getting work published? Or is all this based on some idealized idea of what science "should" be, which conveniently plays in to a narrative that lets you reject anything that doesn't meet your illusory standards as unworthy?

Yes, that patient cherry-picking process is described in this e-mail.

Glen, there are very few websites I can think of - including WoC, which has vastly fewer comments (and far less balance between opposing viewpoints) - that don't have some kind of screening process for comments. More importantly, given that the operating mode of RC is to post rebuttals to skeptical comments inline with those comments themselves, the stuff about "screening through" is not the sinister cherry picking you make it out to be. Likewise, the academic reputation of a journal is damned important, and it is a legitimate point to bring up to the editors of said journal. How seriously would MDs take a publication that was regularly publishing HIV-denialist stuff? Not very, I'm guessing.

Then again, the entire MO of climate skeptics is to endlessly cherrypick - the vindication of Mann in the NRC report after years of attacks from McIntyre et al is meaningless, it's all about today's fresh new scandal. So expecting coherence from y'all on this is probably a bit much.

Your comment about the Russians is really more in your wheelhouse, Glen - why don't you favor us with another long comment showing how progressivism is wholly descended from communism and we can call it a day?

And your basis for saying this is, Robin...? Do you have any relevant experience in conducting research and getting work published? Or is all this based on some idealized idea of what science "should" be, which conveniently plays in to a narrative that lets you reject anything that doesn't meet your illusory standards as unworthy?

@Chris, my basis is that deliberately withholding data is antithetical to science. Period.

Chris:
Likewise, the academic reputation of a journal is damned important, and it is a legitimate point to bring up to the editors of said journal.

They weren't bringing it up with the editors. They don't speak to the editors, because the editors aren't cool enough to be on their cheerleading squad.

They bring it up with the publishers, hoping that their accusation - "WHETHER IT IS TRUE OR NOT" - will prompt them to fire the editors.

So, Chris, do you honestly expect anyone to take your argument from authority seriously?
Setting that to the side, do you maintain that it is standard scientific practice to

evade taxes?

artificially truncate data sets?

violate relevant FOIA laws?

conspire to hide data from one's peers?

artificially alter underlying data?

I ask because a quick scan of the emails shows all of these behaviors. That may be how you practice science, but it certainly isn't reputable.

That's a bullshit argument, chuck. I bring up my experience in science as a way to invalidate AL's argument that this is bad science.

You are arguing from authority using your doctorate as a badge. And yes, good science involves honesty and good scientists are most intrigued by anomalies, for anomalies are the signposts of something not understood. Bad science is airbrushing anomalies, unsupported claims, and hiding embarrassing data. It also tends to be accompanied by poor error analysis and misuse of statistics. And all of the above seem to be the case here.

This crap started back with the ozone problem, where one of the early researchers accused other scientists at the AGU meeting of being immoral for not immediately jumping on the bandwagon and trying to force public policy. I got a report of that from an attendee and he wasn't happy with it. As far as I can see, that political side of science has become dominant in some fields, and it is unfortunate. I think we can be thankful that the JGR and GRL have remained faithful to true science. Nature and Science, high impact journals to be sure, have less to be proud of.

The entirety of Real Climate is basically one long list of them patiently and successfully rebutting the claims of skeptics who insist that the science is flawed, only to be swatted down again and again.

Unfortunately these "successful rebuttals" generally consist of asserting that they are right and the critics are wrong, possibly with some technical sounding jargon thrown in. However having read both sides of several of these controversies, their "rebuttals" simply don't hold water. So maybe if you aren't looking into it too critically you'd be convinced but I'm not. RealClimate is basically a damage-control web site which seeks to reassure people who've already made their minds up that their beliefs are safe.

Take for example the discovery by Steve McIntyre that a number of proxy series were used upside-down by Mann and others. In other words they'd taken some proxy which was said correlated positively to temperature, and then used it in a composite where the algorithm flipped it upside down before including it in the average. They duck and weave and bob, either denying that they used it upside down (when they demonstrably have done so), or saying that "it doesn't matter". Well, it's suprising how many flaws the science can have and it still "doesn't matter". The argument is generally that one flaw doesn't matter since there's other evidence that says the same thing. Only later we find out that that evidence too is flawed. But wait, that doesn't matter either, because... well it can go on forever.

Bottom line: not impressed with this line of reasoning. It suggests a worrying level of defensiveness. Science is an open processed, and supposed to welcome constructive criticism. Something is seriously wrong here.

Chris #31:

Do you have any relevant experience in conducting research and getting work published?

I do, both academic and industrial.

The practices described so far are much more what I would expect of a sales department attached to an engineering corporation, or a turf/ego fight among scientists, than actual science.

The common thread in both the former situations is squabbling over a scarce resource: Market share on the one hand, mind share on the other.

The question of where the line is rightly drawn is not an easy one to answer, despite all the people who will howl full-throated. It is in my experience murkiest when two large engineering organizations go at it, because there is a presumption that both sides know what they're doing and know enough to ask the right questions and can bring the resources to bear and answer them. A free-wheeling caveat emptor mentality is sometimes the best that can be done, although even there, truly faking data or actively conspiring to hide bad data is, shall we say... frowned upon. The legal phrase, "Due diligence," applies in more than one way.

It is less murky when there is a great imbalance between two parties: A multi billion dollar technology company can bamboozle a small start-up, but it ought not to. The two organizations are legal peers, but not moral peers.

And it is less murky still when one party purports interest in the TRVTH rather than in mundane matters of dollars, and no good scientist today would admit to any other motivation. (Neither, unfortunately, would any bad one, and there lies the rub.)

So it is perhaps less murky than you're trying to make it. It deserves a thorough investigation from a neutral party.

Alas, I have no neutral parties with relevant expertise to recommend. The question is too inherently political for that, now, and it has been for years.

To paraphrase Aquinas - Boethius says "The argument from authority is the weakest form of argument"...

To the comments above, I'll add the comment that just because that's science as you do it (and I'd love verification on that if you'd care to share some) doesn't make it good science. Yossarian lives in the labs as well as the military, in my experience (I left a doctoral program at UCLA, and sat on the faculty committee in my Masters program at UCB) - which I'll certainly hold up against yours in this case.

No, my antipathy for Al Gore has nothing to do with my concerns for the very bad way that leading AGW scientists have presented and handled the data that's foundational to their discipline.

Chris, I'm laughing my ass off at this - you have a scientific doctorate?

Even basic reading of AGW literature makes it clear that there are multiple, independent sources of data suggesting AGW, from ice cores to tree rings, from satellite readings to ocean temp buoys.

Chris, those are multiple, independent sources of data that support the concept of GW - not AGW. In addition, we have sources of data that suggest that there is GW on Mars and the moons on Jupiter.

The basis of the claim on AGW is a set of computer models, which have never, to my knowledge been released to be peer-reviewed.

As someone with more than a passing knowledge of statistics and computer modelling (urban econometrics was one of my core disciplines, I can say with some personal confidence that - particularly if I can hide my computations - I can make any data series mean virtually anything. I'll suggest a fair amount of humility is needed when trying to quantify inherently wicked problems.

And there is not now has ever been a 'screening' mechanism for comments at WoC - we do delete comments that violate our comments policy.

And I write about what I write about for reasons that I've also explained multiple times in the last decade. It's not hard to find them.

Marc

There was the hockey stick controversy , since discredited by a National Research Council report, among other reviews.
Care to point out where in that report it was discredited? What was discredited was Mann's use of principal component analysis. See the The Wegman Report

To paraphrase Aquinas - Boethius says "The argument from authority is the weakest form of argument"...

I'd agree with this, except I have absolutely no confidence you can tell the difference between an argument from authority fallacy and using legitimately relevant experience as a subject matter expert to drive an argument.

To the comments above, I'll add the comment that just because that's science as you do it (and I'd love verification on that if you'd care to share some) doesn't make it good science.

AL, I said what I've seen of those emails matches how people in academia, in my experience, talk about and deal with data. I didn't say that's how I do science. And I'd be happy to talk more about my experience with actual research, except that I'm 99% sure the commentariat of WoC would be all to happy to bend over backwards to try to put my professional head on the same pike as the rest of the climate scientists under discussion, so, no thank you.

Yossarian lives in the labs as well as the military, in my experience (I left a doctoral program at UCLA, and sat on the faculty committee in my Masters program at UCB) - which I'll certainly hold up against yours in this case.

Er, aren't your degrees in liberal arts, AL? Not really the same thing as scientific or engineering research.

No, my antipathy for Al Gore has nothing to do with my concerns for the very bad way that leading AGW scientists have presented and handled the data that's foundational to their discipline.

And, as I said before, if your background is in liberal arts, you lack a significant amount of context to condemn these guys. As for "how leading climate scientists presented and handled the data," you'd have more of a leg to stand on if you didn't make some fairly silly comments below showing your limited understanding of climate science, or gave any indication that you realize a leaked email archive is not anywhere near the whole of the story.

Chris, I'm laughing my ass off at this - you have a scientific doctorate?

It's ok, AL - I've been laughing my ass off at your failed political predictions for the past several years, so we're even.

Chris, those are multiple, independent sources of data that support the concept of GW - not AGW. In addition, we have sources of data that suggest that there is GW on Mars and the moons on Jupiter.

A) Considering that most of the anti-global warming crowd is quite willing to deny even that any global warming has occurred at all, this distinction is not quite as meaningful as you might think.

B) Many of the sources of evidence that indicate the planet's been getting warmer are also valid and important contributors to models and research which indicate carbon forcing (i.e. AGW) is to blame for the recent uptick in temperatures. So, again, not as much of a difference as you'd think.

C) Good lord, is this really the level of your exposure to AGW argumentation? There are plenty of reasons we're seeing global warming on Mars, many of which don't have all that much to do with solar forcing. Off the top of my head, there are other planets in the system that have recently evidenced cooling trends, but since I can't dig up a cite for that at the moment, I'll just point out there's some fairly substantial research suggesting the sun is not to blame for recent warming trends . (Server was giving me grief on that last link, but it's accessible through archive.org.)

The basis of the claim on AGW is a set of computer models, which have never, to my knowledge been released to be peer-reviewed.

No. I already cited at least one paper (peer reviewed, I believe) above that deals with the idea of forcings outside of computer models, and there are others. The 4th IPCC report's a fairly good index to these. Moreover, there's been a substantial amount of debate over the years about the relevance and validity of computer models in scientific journals. I don't hold you at fault for not being aware of these, AL - virtually nobody who isn't involved in that particular field would be, and I'm sure as hell not claiming to be a climate expert - but it's foolish to assume that because you and/or your conservative buddies haven't heard of said debate that it hasn't occurred.

As someone with more than a passing knowledge of statistics and computer modelling (urban econometrics was one of my core disciplines, I can say with some personal confidence that - particularly if I can hide my computations - I can make any data series mean virtually anything. I'll suggest a fair amount of humility is needed when trying to quantify inherently wicked problems.

And I'll suggest great heaping truckloads of humility is likewise warranted when comparing what I'm guessing is your poly-sci background to actual physical science, AL. The relatively small exposure I've had to economic computer models has pretty well convinced me we're talking apples and oranges vs. physical climate simulations.

And there is not now has ever been a 'screening' mechanism for comments at WoC - we do delete comments that violate our comments policy.

AL, you guys ban commenters as you see fit, you do delete comments that don't meet your standards (which are enforced differently depending who's got the marshall's hat on) and if you don't have it now, in the past you've definitely had filters enabled that hold comments for moderator review. All of which fits pretty comfortably in the "screening" box, as far as I can tell.

And I write about what I write about for reasons that I've also explained multiple times in the last decade. It's not hard to find them.

AL, your reasons for writing what you are largely irrelevant - your single minded focus on trashing liberals and Democrats and being almost entirely silent about anything their political opposites do completely belies your claim of being any kind of progressive liberal. And whatever your reason for doing so, jumping on the AGW-skeptic bandwagon by parroting empty conservative talking points is just another nail in that coffin.

It was the "hockey stick" (specifically MBH98, although later Mann studies shared many of the same problems) that was discredited. I'm familiar with all of McIntyre's criticisms of that study and they are all valid to some extent. For example there's the issue of tree ring series being used in the study which are not believed to be primarily temperature-related (the Bristlecones), there are issues with autocorrelation, there are problems with the confidence interval calculations, there are issues with PCA centering, there are issues with the RE and R^2 calculations and there is the lack of robustness.

However, by far the most damning criticism of MBH98 is its total failure of the Monte Carlo test. Replace the proxy data with random noise with an autocorrelation structure similar to the proxy data (i.e. red noise) and 99% of the time it will produce a "hockey stick" graph. From random data!

I'm really not sure how an algorithm which does that is defensible. How did it even get past peer review?

Greg F-

Care to point out where in that report it was discredited? What was discredited was Mann's use of principal component analysis. See the The Wegman Report

Page 3 - 4 of the NRC report:

The basic conclusion of Mann et al. (1998, 1999) was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years. This conclusion has subsequently been supported by an array of evidence that includes both additional large-scale surface temperature reconstructions and pronounced changes in a variety of local proxy indicators, such as melting on ice caps and the retreat of glaciers around the world, which in many cases appear to be unprecedented during at least the last 2,000 years. Not all individual proxy records indicate that the recent warmth is unprecedented, although a larger fraction of geographically diverse sites experienced exceptional warmth during the late 20th century than during any other extended period from A.D. 900 onward.

Admittedly, the NRC report is more skeptical of pre-1600 data than Mann's original conclusion, but it is, all in all, a fairly supportive document. Roger Pielke Jr., hardly a gung-ho AGW guy, called it a "a near-complete vindication for the work of Mann et al.".

As for the Wegman report, it itself was never formally peer reviewed (unlike the NRC report). It was explicitly commissioned by two AGW-skeptic members of the House Energy Committee back when Republicans were running the show to discredit Mann, and the committee transcript is actually quite fascinating. I particularly enjoyed Rep. Whitfield's admission early on that even other Republicans thought they were being purely political with this, and Waxman's rather great talk about how ephemeral and irrelevant many of the issues in the Wegman report are to actual climate science.

The Wegman report is not the silver bullet you think it is. Sorry, Greg.

Glen-

They weren't bringing it up with the editors. They don't speak to the editors, because the editors aren't cool enough to be on their cheerleading squad.

They bring it up with the publishers, hoping that their accusation - "WHETHER IT IS TRUE OR NOT" - will prompt them to fire the editors.

Yes, yes, you're spinning this as some horrid suppression campaign as much as you can, but it doesn't really hold water as such. Publishers are just as responsible as editors - and just as legitimate a point of contact - when it comes to the reputation of a publication.

As for "WHETHER IT IS TRUE OR NOT", the portion of the email you omitted (but only AGW proponents cherry-pick, right?) makes it clear that the author firmly believes it to be quite true that von Storch is publishing misinformation disguised as science:

I know about what Matthews has done. He did so without contacting Sarah
or me. He uses a statistical emulation method that can never account for
the full range of uncertainties. I would not trust it outside the
calibration zone -- so I doubt that it can work well for (e.g.)
stabilization cases. As far as I know it has not been peer reviewed.
Furthermore, unless he has illegally got hold of the TAR version of the
model, what he has done can only be an emulation of the SAR version.

Personally, I regard this as junk science (i.e., not science at all).

And as such, he seems to me to be entirely in his rights to point out that A) he does believe it to be junk science, and B) completely outside of an endless he-said-she-said to determine the truth of the matter, the perception of junk science is a legitimate matter of concern to the publisher.

There's a great precis of the "juicy" emails over at the Bishop Hill blog. Go read it, and then decide a bit more how you feel about this...

I feel ... un-trusting. Very un-trusting.

Scientists pronounce all the time on things they have no competence to pronounce on: religion, matters outside their field, and in effect with Active SETI / METI, whether as yet undiscovered aliens are friendly to us.

Why not this too?

Chris says this is normal science as he and many other in his profession understand and practice it. His own pulling of educational rank, along with his move to invalidate what Armed Liberal says simply for political reasons ("your long-standing disinclination to mount any serious critique of conservative interests") is in line with the sort of science he defends, so it makes sense to take him at his word.

And, oh yes, Chris is right not to discuss his work. Lots of people would start pretending that he's the only scientist who does business this way. Why invite people to be hypocritical?

But enough about Chris, even though he's been making it all about him and his credentials.

The attempted political restructuring of the global economy, greatly to the detriment of America, the Anglosphere and the West, on the basis of the sort of science best upheld by getting an editor ousted if he doesn't agree with you, is more important.

As for "WHETHER IT IS TRUE OR NOT", the portion of the email you omitted (but only AGW proponents cherry-pick, right?) makes it clear that the author firmly believes it to be quite true that von Storch is publishing misinformation disguised as science:

Beneath contempt.

Well, Chris, one easy way to tell the difference between an argument from authority and an authoritative argument is that the authoritative argument contains arguments, not just credentials.

But that's just me.

For myself, I started out in the physical sciences, and briefly studied with Matt Sands. For a good precis of my reaction to your claims, may I simply refer you and everyone else to Feynman's lecture and article on Cargo Cult Science as a good example of what I see as the kind of issues with the scientific method as you see it - and as it seems to be revealed in the correspondence above.

I'm not suggesting that Soros or Gore sat around and funded scientists to lie; but I will suggest that there is immense institutional pressure to conform in science today, and that pressure directly contradicts the principles of good science as laid out by Huxley or Feynman - two scientists who I have studied and hold in immense intellectual regard.

I'll skip over the ad-hominem political claims and simply suggest that you need to read what I write more thoroughly.

And finally, we have never 'screened' comments at Winds. Our software will flag and hold comments with excessive links or other indicia of comment spam; but I have never - ever - blocked a comment or banned a commenter for the substance of the arguments they have made. You're insulting me and lying to suggest that I - or any of the marshalls who I direct - have or would do so.

Marc

Chris says this is normal science as he and many other in his profession understand and practice it.

David, I said that this is how people in my experience talk about and deal with data. To be more precise, what I've read of the emails is in line with informal terminology and practical day-to-day analysis of data that I've observed when watching scientists and engineers attempt to wrangle a meaningful conclusion out of messy and contradictory data points.

I suspect, however, when you say "this is normal science", you have visions of a mass conspiracy for "political restructuring of the economy, greatly to the detriment of America, the Anglosphere and the West." To be clear, I have not seen, am not engaged in, and do not believe the emails to indicate that scientists are engaged in that kind of action.

Not that I expect you to believe me, but I'm stating it for the record.

His own pulling of educational rank, along with his move to invalidate what Armed Liberal says simply for political reasons ("your long-standing disinclination to mount any serious critique of conservative interests") is in line with the sort of science he defends, so it makes sense to take him at his word.

Actually, David, you've got it entirely backwards - I was explicitly referencing AL's own tendency to attack or ignore things based purely on political motivations, rather than on logic. And I think if you look at my entire quote in context (not that conservative commenters on Winds of Change would cherry-pick or anything - only evil liberal scientists do that, right?) that's rather clear... but eye of the beholder and all that.

And as for "pulling of educational rank," and your insistence that I'm "making it all about me and my credentials," I'd invite you to review what exactly does and does not constitute an appeal to authority fallacy and determine if there's ever a legitimate situation where someone might cite their own experience as a legitimate part of an argument.

On the other hand, in your own words, you feel "un-trusting" of scientists - not just with regard to global warming - and seem pretty convinced they're engaged in a conspiracy to destroy all that's good and Christian in the world. (At least, that's the conclusion I draw based on what I've read of your previous comments regarding "the Anglosphere" vs. everyone else).

So just out of curiosity, David, what do you believe the scientific process should look like? What areas should science not be permitted to explore - not climate change or extraterrestrials or anything that might impinge on religion, based on what you've said above. Shouldn't science be more controlled? After all, climate change guys are trying to destroy the economy, apparently, and we've heard for decades that evolution is undermining people's belief in God. That being the case, do you think the last Bush administration had it pretty much right - that political appointees should be placed in charge of organizations like NASA and the EPA, and basically shape the production of scientific knowledge to something that conservatives like... well, like those on this website, would feel more comfortable with?

And how should scientists be prohibited from trying to police what does and doesn't count as legitimate science - clearly you don't like what some of the emails are saying about complaining to publishers about AGW, but am I wrong in assuming that you'd also like to see biologists prohibited from complaining that Intelligent Design doesn't belong in biology journals? Or geologists complaining about "Young Earth" research mixing it up with their precious model of a 4+ billion year old planet? Presumably as long as you can find an editor who's willing to give things a hearing - Hollow Earth, vaccines-cause-autism, HIV-doesn't-cause-AIDS - then we should entertain just about anything in peer-reviewed, scientific debate, and scientists who feel differently can spend their time pushing back against all those arguments, rather than spending their time conducting new research.

Because now that I think about it, it really is far more important that we get to the bottom of this attempt to politically restructure economy, and as you've identified yourself as someone properly un-trusting of scientists, I really am curious what your plan is to put us all in our place.

Chris:

Not that I expect you to believe me, but I'm stating it for the record.

Stating things for the record is good.

Chris:

And as for "pulling of educational rank," and your insistence that I'm "making it all about me and my credentials," I'd invite you to review what exactly does and does not constitute an appeal to authority fallacy ...

You shifted ground. There's far more to the use of credentials to win an argument than the formal fallacy of the appeal to authority alone.

And - for the record - of course there are legitimate situations where someone might cite their own experience as a legitimate part of an argument.

Chris:

And as for "pulling of educational rank," and your insistence that I'm "making it all about me and my credentials," I'd invite you to review what exactly does and does not constitute an appeal to authority fallacy ...

You shifted ground. There's far more to the use of credentials to win an argument than the formal fallacy of the appeal to authority alone.

Chris:

On the other hand, in your own words, you feel "un-trusting" of scientists - not just with regard to global warming - and seem pretty convinced they're engaged in a conspiracy to destroy all that's good and Christian in the world. (At least, that's the conclusion I draw based on what I've read of your previous comments regarding "the Anglosphere" vs. everyone else).

Ultra-bargain basement cheap shot. You're just putting bad words in my mouth there.

Chris:

So just out of curiosity, David, what do you believe the scientific process should look like?

Less like a guild where scientists say what it takes to increase their funding (or further other political, religious or personal goals they may have), hide their doubts from the ignorant populace, close ranks when challenged, make it as hard as they can for skeptics to get hold of the facts needed to challenge them, and raise the costs of disagreeing, by means ranging from ridicule to going after people's jobs. I'd like the process to be less conspiratorial, much more modest, and more empirical than political. I'd like it to be very easy to find neutral scientists to clear up a wide range of disputes. I'd like scientists to feel no more qualified to pronounce on religion, philosophy and the minds of aliens than even the very best auto mechanics do.

OK, can someone chop the repeated paragraph there, and this line? Sorry.

Chris:

What areas should science not be permitted to explore - not climate change or extraterrestrials or anything that might impinge on religion, based on what you've said above.

No, not based on what I said above. You're putting bad words in my mouth again, and playing ground-shifting games again.

Why do you keep doing that? You're not fooling people. You're obviously being rude and silly. If it's that raising the heat of a conversation is a tactic to leverage credentials by making it harder to have a polite, low-stakes disagreement in neutral language, people have already notice what you are up to there.

I think that's enough. The importance of the conspiracy of scientists that has just been revealed entirely outweighs any interest or importance that may attach to me, just as it entirely outweighs any interest or importance that may attach to you.

Well, Chris, one easy way to tell the difference between an argument from authority and an authoritative argument is that the authoritative argument contains arguments, not just credentials.

Right, and I haven't provided any arguments, have I? No links to original research in my comments, or citations of relevant findings from investigatory bodies. I certainly haven't a counter-argument to your assertion that AGW is questionable because we've seen evidence of warming on Mars. Because you'd well know if somebody was making an argument, and you'd presumably want to continue bouncing that argument back and forth to arrive at something approaching a mutually agreed-on truth, rather than ignoring it entirely so you can stick to your existing notions about AGW science being flawed.

But since I haven't made any such argument, then you're on solid ground!

I'll skip over the ad-hominem political claims and simply suggest that you need to read what I write more thoroughly.

Back at you, AL.

For myself, I started out in the physical sciences, and briefly studied with Matt Sands.

Fantastic. But you seem to be dancing around the issue that you don't actually have degrees in any kind of scientific discipline, and I assume if you had any kind of in-depth experience, you'd have mentioned it by now. If you want to embrace the kind of honesty Feynman talks about, I'd suggest you just admit that fact, and move forward.

For a good precis of my reaction to your claims, may I simply refer you and everyone else to Feynman's lecture and article on Cargo Cult Science as a good example of what I see as the kind of issues with the scientific method as you see it - and as it seems to be revealed in the correspondence above.

And I'd suggest, AL, that you're confusing the day to day production and analysis of data- which is what I'm talking about, and what I see in those emails - with the formal mechanism of scientific communication via publications, which is where the relentless honesty Feynman talks about comes in.

Let's also keep in mind that different scientific disciplines are of necessity not going to be as utterly rigorous as Feynman's world of experimental sub-atomic physics, where experiments are inherently more controllable (and where, in the 50's and 60's when Feynman did a lot of his most important work, federal money was almost unlimited). There are numerous disciplines - biology, astronomy, climatology - where experimental parameters are at least partially beyond the experimenter's control. We recognize that as being an unavoidable aspect of the field and move on, rather than writing off every experiment that isn't utterly perfect as being junk.

And finally, we have never 'screened' comments at Winds. Our software will flag and hold comments with excessive links or other indicia of comment spam; but I have never - ever - blocked a comment or banned a commenter for the substance of the arguments they have made. You're insulting me and lying to suggest that I - or any of the marshalls who I direct - have or would do so.

Are we really going to get into semantics here, AL? You sure as hell have closed comment threads because you don't like where they're going, politically. I largely stopped commenting here because one particularly memorable thread was closed, then reopened but with an arbitrary 300-word limit enforced on everybody but the marshall in charge, who was free to continue spouting conspiracy theories about William Ayers to his heart's content. Those aren't lies, they're facts. And yes, you do hold comments to check for spam - guess what, that's a screening process, even if it doesn't involve any kind of political bias.

The bigger and more important question, to my mind, is whether RC's comment policy is inherently biased by blocking or deleting comments that the site owners just don't like, and the wording in the above email is ambiguous on that matter. On the other hand, the RC guys do an admirable job of responding in detail, point by point (whatever you think of the content of their reply) to their critics. It's a model other websites, including this one, might consider.

Glen-

Beneath contempt.

Ditto, buddy! Have a good one!

David-

You shifted ground. There's far more to the use of credentials to win an argument than the formal fallacy of the appeal to authority alone.

And - for the record - of course there are legitimate situations where someone might cite their own experience as a legitimate part of an argument.

Super. So you can explain where I'm falling on that scale and why, or we can just drop this line of argument.

No, not based on what I said above. You're putting bad words in my mouth again, and playing ground-shifting games again.

Why do you keep doing that? You're not fooling people. You're obviously being rude and silly. If it's that raising the heat of a conversation is a tactic to leverage credentials by making it harder to have a polite, low-stakes disagreement in neutral language, people have already notice what you are up to there.

Ok, David, let me make a serious try to get this across:

- I did not find your comments #44 and 45 to be polite or low-stakes. Rather than arguing about whether you intended them to be that way or not, let's just take it for granted that they did not come across that way, and move on.

- You say I'm putting words in your mouth, but from my perspective, I have at least as many reasons to believe what I wrote about you as you have to believe that there's some grand conspiracy of scientists out to destroy the well-being of the English-speaking people of the world. If you're going to interpret the worst possible reading to a bunch of people based on an incomplete record of context-intensive communications that were never meant to be publicly viewed, then you will, in turn, get the worst motives and the most simplistic ideas assigned to you in turn, based on your far more straightforward and obvious comments. If, on the other hand, you take a step back and communicate an analysis of what you've seen that evidences some degree of nuance and open-mindedness, you'll get the same in return.

That said, trying to bring that degree of nuance and open-mindedness to your proposals above:

[David wants the scientific process to behave..] Less like a guild where scientists say what it takes to increase their funding (or further other political, religious or personal goals they may have),

Scientists are humans working in a human institution. In my experience, many of them are genuinely and demonstrably less interested in power games than they are about their research, but there has never been any kind of human structure where personal desires did not creep in at some point. Expecting otherwise will get you nowhere, and is an unfair standard to hold scientists to. Rather than assume you don't haven't worked with scientists in any kind of professional capacity, David, I'll simply suggest that, if you don't know any scientists in a professional capacity, that you should ideally become more familiar with that world before you make judgment calls over how much their personal ambition colors their work. And if you do know scientists in a professional capacity and still come to the conclusions you have, I'll just register my genuine surprise that you feel as you do, and leave it at that.

hide their doubts from the ignorant populace,

Again, I have not seen this specifically w/r/t climate scientists. Insofar as it is an issue, it's something I've seen far more often in the intersection of the corporate world, where scientific doubts and misgivings about new technologies and medications (especially the latter) are swept under the rug in favor of an immediate corporate profit motive. That said, I do think that business execs and marketing folks are far more to blame for this at a corporate level, so it's all kind of a wash, from my perspective, when it comes to scientists.

close ranks when challenged,

How much of that is actual herd mentality, and how much of it is a scientific community legitimately wanting to focus on the areas of contention they themselves are aware of as active issues, vs. having to go back and retread over controversies that were settled in the literature years ago?

make it as hard as they can for skeptics to get hold of the facts needed to challenge them,

I will point out in the ideal scientific world AL is pushing, skeptics could and should simply reduplicate the relevant research themselves - that is, make themselves legitimate peers in the scientific process. That said, I'd probably agree that degree of openness could probably be a good thing, if there is instituted some mechanism where a scientific researcher is not obligated to completely reprove his entire body of work to every new skeptic that comes by.

and raise the costs of disagreeing, by means ranging from ridicule to going after people's jobs.

I believe there are times, depending on the context, where ridicule or going after people's jobs is entirely the right moral and ethical choice to make. That said, this is obviously something that would need to be dealt with on a case-by-case basis, and insofar as this email leak may demonstrate such cases, that's a good thing.

I'd like the process to be less conspiratorial, much more modest, and more empirical than political.

I personally find it to be none of those things in any great measure, and, assuming you are not, I invite you to become more thoroughly involved in the process to decide for yourself. That said, my above point about humans operating in a human institution bears repeating.

I'd like it to be very easy to find neutral scientists to clear up a wide range of disputes.

This is inherently a difficult problem, because anyone who knows enough to be useful in an area of contention almost certainly has his or her own opinions on the subject. That said, if you can suggest some mechanism for this, I'd like to hear it.

I'd like scientists to feel no more qualified to pronounce on religion, philosophy and the minds of aliens than even the very best auto mechanics do.

I think this is a fundamentally flawed view of science and to the people involved in it. Science is inherently curious - Feynman certainly did not limit his intellect to particle physics, and likewise, neither do most scientists I know limit themselves to their immediate work. Of course, you do occasionally get someone like Richard Dawkins who's not only way outside of what can be called his area of expertise, but is quite obnoxious about it. That said, that's largely an anomalous situation, and given what we know about the origins of creativity and innovation (that it is frequently the result of insights in one domain being transferred to another) having scientists who are constantly pushing at all boundaries of knowledge almost certainly brings us more benefit than if they just kept their heads down and went home at the end of the day.

David-

You shifted ground. There's far more to the use of credentials to win an argument than the formal fallacy of the appeal to authority alone.

And - for the record - of course there are legitimate situations where someone might cite their own experience as a legitimate part of an argument.

Super. So you can explain where I'm falling on that scale and why, or we can just drop this line of argument.

No, not based on what I said above. You're putting bad words in my mouth again, and playing ground-shifting games again.

Why do you keep doing that? You're not fooling people. You're obviously being rude and silly. If it's that raising the heat of a conversation is a tactic to leverage credentials by making it harder to have a polite, low-stakes disagreement in neutral language, people have already notice what you are up to there.

Ok, David, let me make a serious try to get this across:

- I did not find your comments #44 and 45 to be polite or low-stakes. Rather than arguing about whether you intended them to be that way or not, let's just take it for granted that they did not come across that way, and move on.

- You say I'm putting words in your mouth, but from my perspective, I have at least as many reasons to believe what I wrote about you as you have to believe that there's some grand conspiracy of scientists out to destroy the well-being of the English-speaking people of the world. If you're going to interpret the worst possible reading to a bunch of people based on an incomplete record of context-intensive communications that were never meant to be publicly viewed, then you will, in turn, get the worst motives and the most simplistic ideas assigned to you in turn, based on your far more straightforward and obvious comments. If, on the other hand, you take a step back and communicate an analysis of what you've seen that evidences some degree of nuance and open-mindedness, you'll get the same in return.

That said, trying to bring that degree of nuance and open-mindedness to your proposals above:

[David wants the scientific process to behave..] Less like a guild where scientists say what it takes to increase their funding (or further other political, religious or personal goals they may have),

Scientists are humans working in a human institution. In my experience, many of them are genuinely and demonstrably less interested in power games than they are about their research, but there has never been any kind of human structure where personal desires did not creep in at some point. Expecting otherwise will get you nowhere, and is an unfair standard to hold scientists to. Rather than assume you don't haven't worked with scientists in any kind of professional capacity, David, I'll simply suggest that, if you don't know any scientists in a professional capacity, that you should ideally become more familiar with that world before you make judgment calls over how much their personal ambition colors their work. And if you do know scientists in a professional capacity and still come to the conclusions you have, I'll just register my genuine surprise that you feel as you do, and leave it at that.

hide their doubts from the ignorant populace,

Again, I have not seen this specifically w/r/t climate scientists. Insofar as it is an issue, it's something I've seen far more often in the intersection of the corporate world, where scientific doubts and misgivings about new technologies and medications (especially the latter) are swept under the rug in favor of an immediate corporate profit motive. That said, I do think that business execs and marketing folks are far more to blame for this at a corporate level, so it's all kind of a wash, from my perspective, when it comes to scientists.

close ranks when challenged,

How much of that is actual herd mentality, and how much of it is a scientific community legitimately wanting to focus on the areas of contention they themselves are aware of as active issues, vs. having to go back and retread over controversies that were settled in the literature years ago?

make it as hard as they can for skeptics to get hold of the facts needed to challenge them,

I will point out in the ideal scientific world AL is pushing, skeptics could and should simply reduplicate the relevant research themselves - that is, make themselves legitimate peers in the scientific process. That said, I'd probably agree that degree of openness could probably be a good thing, if there is instituted some mechanism where a scientific researcher is not obligated to completely reprove his entire body of work to every new skeptic that comes by.

and raise the costs of disagreeing, by means ranging from ridicule to going after people's jobs.

I believe there are times, depending on the context, where ridicule or going after people's jobs is entirely the right moral and ethical choice to make. That said, this is obviously something that would need to be dealt with on a case-by-case basis, and insofar as this email leak may demonstrate such cases, that's a good thing.

I'd like the process to be less conspiratorial, much more modest, and more empirical than political.

I personally find it to be none of those things in any great measure, and, assuming you are not, I invite you to become more thoroughly involved in the process to decide for yourself. That said, my above point about humans operating in a human institution bears repeating.

I'd like it to be very easy to find neutral scientists to clear up a wide range of disputes.

This is inherently a difficult problem, because anyone who knows enough to be useful in an area of contention almost certainly has his or her own opinions on the subject. That said, if you can suggest some mechanism for this, I'd like to hear it.

I'd like scientists to feel no more qualified to pronounce on religion, philosophy and the minds of aliens than even the very best auto mechanics do.

I think this is a fundamentally flawed view of science and to the people involved in it. Science is inherently curious - Feynman certainly did not limit his intellect to particle physics, and likewise, neither do most scientists I know limit themselves to their immediate work. Of course, you do occasionally get someone like Richard Dawkins who's not only way outside of what can be called his area of expertise, but is quite obnoxious about it. That said, that's largely an anomalous situation, and given what we know about the origins of creativity and innovation (that it is frequently the result of insights in one domain being transferred to another) having scientists who are constantly pushing at all boundaries of knowledge almost certainly brings us more benefit than if they just kept their heads down and went home at the end of the day.

I'm not interested in these moves to change the issue. Lets get back to the much more interesting topic of what the emails reveal.

I think the guys at PowerLine are doing a good job of explaining this (link) (link). They can do so because it's really a matter of systematic fraud. It's not unfamiliar stuff for lawyers.

OK, first, I am jealous of Marc for getting to study, however briefly, with Matt Sands.

Second, why have we regressed so far in technology? In USENET days, we could have killfiled Chris once it became clear that he was trying to misdirect the conversation, and was beyond rational argument, and saved ourselves a lot of time and pixels.

Third, what really bothers me about the whole AGW argument, and has since the beginning, is the cabalistic and cultish nature of the AGW promoters.

I'll address the 'mystery religion" aspect first. If the data cannot be produced, and if the models cannot be openly examined, then the conclusions are worthless, because the results cannot be reproduced. Good science, whatever Chris might think, requires the ability to get at the data and methods; you have to show your work. Good example: after the (Michaelson-Morley?) experiments that showed that light's apparent speed was not dependent on orientation, some guys whose names and categories I've now forgotten (I think they were physical scientists) theorized that this could only be true if the length of one of the arms in the MM experiment was changing due to motion. They had no scientific basis for this; it was just a hunch. Their conclusion was thus ignored until about ten years later, when Einstein came out with special relativity. If the MM experimenters had not described their apparatus and detailed their methodology and results, Einstein would have lacked the starting point he used for asserting special relativity. So when I hear that the AGW data is lost, that the models are proprietary and not to be shared, and that we should just "trust them," with them being the high-poobahs of the AGW promotion camp, my automatic response is, "prove it."

As to the cabalistic nature of the AGW promotion, consider Chris' points above, when he was briefly trying to make an argument, a good example. You can't argue with data set 1, because it agrees with data set 2, which you haven't argued with. You can't argue with data set 2, because it agrees with data set 3, which you haven't argued with. You can't argue with data set 3, because it agrees with data set 1, which you haven't argued with. It's self-referential, and all the supposed data and methodologies (if they won't release them, then as far as I'm concerned they've made them up out of whole cloth) come from a very small number of activists who support each other. There's no outside debate really allowed, because any time someone (say, McKittrick) comes along and argues with them, it is dismissed as not taking on the whole issue at once and not being properly scientific and so forth; the actual objections and arguments are ignored. Worse still, there is significant evidence in these leaked emails that the AGW proponents' strongest argument from authority (the critics haven't had their work published in peer-reviewed journals) is based on a fraud: the AGW promoters appear to have actively conspired to prevent that publication.

So I have never been convinced that the earth is warming overall, though I'm willing to be if someone can produce good evidence. I've certainly never been convinced that any such warming is caused by man's actions, though again I'd be willing to look at the evidence and change my mind if needed. And every time I attempt to find someone to convince me, they descend into the kind of morass that Chris jumped into, the moment that I start asking questions.

As far as I'm concerned, these leaked bits are at best merely confirming what was already evident: AGW promotion is a sham and a fraud perpetrated by activists with scientific credentials, rather than meaningful science properly conducted, and its aims are entirely political.

Time Oren: I'll just note that given the time interval in question, the e-mails can only be a small fraction of those sent and received. So they must be cherry picked in some sense - not saying whether good or bad. The original hacker/leaker called this a 'random selection' so there may well be more than will / could come out. Someone may be learning from Breitbart's tactics in dribbling out his most damning evidence.

The emails are so focused on the issue that I assume that the "random selection" comment is sarcasm. It seems much more likely that the data was compiled in response to an FOIA request, perhaps the one from Steve McIntyre which was denied on November 13, 2009. The fact that the most recent email was dated November 12, 2009 is either very revealing or a major coincidence. If it was compiled for an FOIA request, then it is much more likely that one of the people involved in assembling the data decided that he (or she) could not be associated with a coverup than it is that a random hacker just happened to pick that time to break into the CRU computer system and find the files.

Mark Beuhner: This is crazy. The data doesn't match the models therefore something must be wrong with the data? What kind of science is that?

But that seems to be a common thread in the AGW work. Radiosondes carry thermometers to measure the temperature profile of the atmosphere as they ascend. Those temperature profiles measured near the equator have failed to detect the signature that all of the computer models agree must be present if CO2 or other greenhouse gases are causing a temperature increase on the surface. So some AGW advocates have proposed rejecting the thermometer data and using the wind speed profile as a surrogate means of measuring the temperature, arguing that this is somehow a more accurate temperature sensor than devices specifically designed and universally used to do precisely that.

Group think. That is what these emails scream.

Despite what Chris is arguing, scientists need to stand by their work, make their data and analysis available to be double checked and replicated. Even arguing that that is not the case is a huge red flag.

What really screams group think is that any time a particular experiment has questions, part of the rebuttal is about 'all the other experiments'. That is IMMATERIAL. If there is a problem with tree rings, ice cores can't be brought up as a defense. And if there is a problem with ice cores, satellites can't be brought up. That kind of thinking creates a circular logic, the results of which we are seeing. Quite likely it creates a house of cards with the data, where each pillar of the overall theory depends on the supposed strength of others which may be equally in question.

Chris: That said, the idea that Al Gore "singlehandedly legitimized AGW" is ludicrous, as is the implication that he's somehow manufacturing a crisis to get rich. It's simply a conspiracy theory of the highest order, and it's frankly a sign of how vastly out of whack AL has gotten that he'd seriously make that argument.

Al Gore has managed to win both a Nobel Peace Prize, shared with the IPCC, and an Oscar for his advocacy on AGW. If Cap-and-Trade is enacted, there is every reason to expect that carbon trading will become the largest commodity market in the entire economy. It is no mystery why Enron was pushing so hard for this policy, before the collapse of the company.

Chris: Are there cases where valuable data has been lost? Absolutely - that's not at all uncommon in science. (We taped over the freaking MOON LANDING , remember?) But if you actually follow the back and forth of climate debate - I have, and I'm very skeptical you've done the same outside of skimming through conservative talking points on the subject, based on your arguments here - then it's hard to come to the conclusion that there's somehow been some withholding of vital information in the AGW debate. Even basic reading of AGW literature makes it clear that there are multiple, independent sources of data suggesting AGW, from ice cores to tree rings, from satellite readings to ocean temp buoys. The idea that finding some kind of smoking gun with one of these areas of research will somehow undermine all of AGW theory is misinformed; the idea that the underlying data supporting AGW hasn't been properly examined is, at best, ignorant, and at worst willfully foolish.

This is a case where valuable data has been lost, and we have emails in which the person who has custody of the data is telling his friends that he will destroy it before he shares it with his critics.

There are multiple independent sources of data suggesting GW, but the group of people generating this data is small and all of them seem to have this phobia about letting anyone outside of their group see the raw data. Most scientific journals, and many funding organizations, have data archival requirements that the Team has been allowed to ignore. Lonnie Thompson has been collecting ice cores for decades, and has never archived any of his data, even though some of it is absolutely irreplaceable.

Every source of data needs to be capable of withstanding scrutiny. And those Team members who reject examination of the raw data and the analysis techniques by outsiders reject the reason that the scientific method is self-correcting.

Jeff-

Second, why have we regressed so far in technology? In USENET days, we could have killfiled Chris once it became clear that he was trying to misdirect the conversation, and was beyond rational argument, and saved ourselves a lot of time and pixels.

For someone you'd like to killfile, Jeff, you're doing an awfully good job of arguing me via backhanded remarks in the rest of your post. That said, believe me, rest assured the lack of respect goes both ways.

As to the cabalistic nature of the AGW promotion, consider Chris' points above, when he was briefly trying to make an argument, a good example. You can't argue with data set 1, because it agrees with data set 2, which you haven't argued with. You can't argue with data set 2, because it agrees with data set 3, which you haven't argued with. You can't argue with data set 3, because it agrees with data set 1, which you haven't argued with.

I never made this argument, Jeff. I have said - and AL seems to agree - that there are multiple independent data sources showing global warming, and that disproving one of those sources in and of itself isn't sufficient to disprove global warming. But the sources are independent, they can and should be dealt with independently, and I defy you to find a place in my comment where I said differently.

So I have never been convinced that the earth is warming overall, though I'm willing to be if someone can produce good evidence. I've certainly never been convinced that any such warming is caused by man's actions, though again I'd be willing to look at the evidence and change my mind if needed. And every time I attempt to find someone to convince me, they descend into the kind of morass that Chris jumped into, the moment that I start asking questions.

I'm going to go out on a limb here and suggest that you've never actually bothered to read through the IPCC, Jeff, or any actual science produced on the subject. Certainly the anti-AGW commenters on this thread have done a great job at condemning AGW science without virtually any references to any actual published work in the field, preferring to take the worst interpretation of these emails as evidence that all climate scientists, everywhere, are frauds, and therefore unworthy of any engagement.

Which is about as good an example of a self-reinforcing conspiracy theory as I can imagine, even as you pat yourself on the back about how fair-minded you are, and how you'd be willing to listen to actual proof on the subject if anyone trustworthy would come along and explain it to you without making it so gosh-darn complicated.

Kudos, Jeff. You're a true exemplar of the intellectual characteristics of your political cohort.

chuck: Oh, wow. Hat's off, gentlemen, a genius. Any of you other ignorant yokels here have a doctorate?(sheepishly raises hand).

I have a Ph.D. in Astronomy, from a top-5 university. I've spent the last three decades working in the aerospace industry, but one of my published papers is one of the top 300 most-cited papers in astronomy in the past century. This is because it not only provided useful analysis results but also shared the raw data so that other research groups could extend the analysis. As a result, the paper is foundational in a whole branch of astronomy, and will be more heavily cited this year than in any previous one.

That is the way that I think any science should be done.

The emails show behavior that is antithetical to science. I'm not sure whether it reminds me more of junior high schoolgirls or a criminal conspiracy, but it is not science.

Chris, let me see if I can wave aside the sandstorm you've kicked up and get us down to a few questions.

As to argument from authority - no, Chris, you've offered nothing to justify the statement that "this is typical science" except your claim to have BTDT. I'm amazed that you didn't open with "No shit, there I was..." There are vast bodies of research on the culture of science, how science is actually practiced, etc. and I'd assume you'd have some of those available to you. I'm making the claim that the patterns of deceitful behavior shown in the (purported) emails is atypical and is - intellectually - bad science. You're making the claim that it's typical. I'd love to see citations on that, which would in fact move us past your claim of authority on the subject into the realm of argument.

As to whether what's been shown is intellectually good science, let's be a little reductive.

All of the work that I'm aware of in the AGW field (and I've read some, but not nearly a representative sample) involves taking datasets - either contemporaneously gathered datasets (satellite or local temperature readings, various climate measurements like air pressure, humidity, etc.) or historically reconstructed datasets (built from proxy data like tree growth or sampled directly like ice gases from deep cores) and applies various mathematical techniques to the data, typically embodied as computer code.

Now I hold as a core value of science - as it was taught me as a wee lad - that it's intersubjective; i.e. it doesn't (outside the context of specific experiments) matter who's looking, but rather that experiments are reproducible by anyone.

As a student of Horst Rittel and Mel Webber (OK, OK, a small claim of authority on my part, to be sure) who co-wrote the original paper on 'Wicked Problems,' I'm deeply familiar with the problem that hugely complex systems - like economies or climate systems - present to the traditional scientific method.

But in no case - while Rittel and Webber devalue the usefulness of traditional scientific method is dealing with wicked problems - does their work or any one else's with whom I'm familiar suggest that claims of scientific rigor do not need to be attached to simply reproducible results.

Now to reproduce results in AGW, what would then be required would be that the raw datasets and process - including computer code where used - be released as a part of the notes to any 'experiments' that were conducted using those datasets.

That's standard practice in economics journals that I'm aware of; and yet - the core data underlying the foundational papers in the disciple supporting AGW have not been so released.

So here's a direct, yes or no set of questions to you to help all of us understand a little better how you see this.

1) Do you believe reproducibility matters in science?

2) Where the science is based on observational data and computation, would you support concealing access to that data and those processes?

and finally...

3) Have the core AGW datasets and code been released for general review, and if so where?

If I'm wrong, and the code and data has been released, I'll be happy to post a picture of a large crow as the banner on this site for a month.

Marc

David-

I'm not interested in these moves to change the issue. Lets get back to the much more interesting topic of what the emails reveal.

sigh So much for wanting to pursue a "polite, low-stakes disagreement in neutral language," eh, David?

Go ahead and let us know the depth of the conspiracy you've uncovered, and then I'm sure we'll get on to the far more interesting bits of how you propose to shape the scientific process to stop scientists unduly opining on "religion, philosophy and the minds of aliens."

I was going to respond to this:

I'm going to go out on a limb here and suggest that you've never actually bothered to read through the IPCC, Jeff, or any actual science produced on the subject. Certainly the anti-AGW commenters on this thread have done a great job at condemning AGW science without virtually any references to any actual published work in the field, preferring to take the worst interpretation of these emails as evidence that all climate scientists, everywhere, are frauds, and therefore unworthy of any engagement.

until I got to this:

Which is about as good an example of a self-reinforcing conspiracy theory as I can imagine, even as you pat yourself on the back about how fair-minded you are, and how you'd be willing to listen to actual proof on the subject if anyone trustworthy would come along and explain it to you without making it so gosh-darn complicated.
Kudos, Jeff. You're a true exemplar of the intellectual characteristics of your political cohort.

You see, your use of ad hominem is only part of the reason why it is futile to argue with you. I was not, in my prior post, intending to argue with you. Instead, I was using your comments as an illustration of a problem with the proponents of AGW: the tendency to resort immediately to character attacks (assuming I cannot handle complicated subjects, for example), impugning motives rather than presenting arguments, and making poor assumptions of the preparation of one's adversaries in argument (I have, in fact, read the IPCC report, and found it vastly wanting as science, and remarkably good as political theater). So in a sense, you've really just given a better illustration of these points with your response than you did with the material I was originally referencing. You seem reasonable, at first, and then retreat into aggressively push fallacies and complete irrelevancies the moment that even mild criticism is advanced, or even well-intentioned questions are asked.

And that's fine; it's certainly your right, and one poor argument on a blog is hardly going to convince me one way or the other. But it's worth noting that I've not yet seen any behavior but this from the AGW proponents. (You get the same from some of the peak oil guys, but not all of them.) And that alone is suggestive of the possibility that the AGW hypothesis is a load of bunk: bad faith arguments suggest that something is being hidden. Actually hiding the data is direct evidence of exactly that. These releases strongly indicate that in fact data was being deliberately hidden, in complete bad faith.

You can dance around that all you want, but for any rational person who's not politically committed to either side (that would include me) of the debate, this just screams fraud on a grand scale. In fact, I am far less willing to be convinced than I was a week ago, and somewhat less willing to be convinced than I was at the start of this thread.

Chris, i'll make this even simpler:

Is destroying data rather than releasing it for review part of the typical scientific process?

Recently, on another forum, I was having a debate with a couple of people who are proponents of intelligent design. I started with, "OK, convince me," and then asked questions about their arguments. It got to the point where I gave a set of criteria for something to be considered scientifically valid, and later pointed out the places where ID does not fit. The response was almost exactly of the kind Chris keeps giving: ad hominem attacks; references to authority as a way of diverting from rational arguments; attempts to change the subject to matters not relevant to the topic at hand; and arguments that words don't have the meanings that I, or some text book, or even the dictionary might assign to them. I just got a giggle when I realized that Chris' argument strategy parallels that of the ID proponents.

Here, by the way, are the criteria I gave:
Evolution is a scientific theory, by which I mean that it satisfies all the fundamental criteria necessary to apply the scientific method to answer the question. These include:
  • The theory is falsifiable: there is evidence that, if it could be produced, would invalidate or require modification of the theory.
  • The theory proposes natural processes to explain natural phenomena.
  • The theory makes definite predictions that may be tested by observation and/or experiment.
  • The observations and experiments that demonstrate the theory are repeatable.
So one of two things is true: either ID also meets these criteria, in which case it is a scientific theory in its own right and may be judged true or false; or it does not meet these criteria, which means it is based on some other way of knowing.

AL-

1) Do you believe reproducibility matters in science?

Yes, AL, and I have never said otherwise.

2) Where the science is based on observational data and computation, would you support concealing access to that data and those processes?

No, I would no support concealing access to that data.

That said, there are, in academia, multiple hurdles and hassles involved with publicly releasing data and code due to funding agreements, intellectual property agreements, etc. Just because data is not instantly and universally released does not automatically imply scientific misconduct.

Moreover, I can certainly understand the frustration, evidenced in these emails, of climate scientists constantly harassed by would-be scientists (e.g. McIntyre) who seem far more interested in discrediting their ideological opponents than in determining the truth of an issue. The key issue is whether, as GregC mentions above, the scientists in question have actually destroyed the data in question, or whether they're just blowing of steam in an email to colleagues , akin to somebody in a corporate situation saying "we're going to murder those guys."

3) Have the core AGW datasets and code been released for general review, and if so where?

AL, I predict this will turn into a rather tedious argument about what "the core AGW datasets" actually encompasses. Moreover, different groups and orgs and journals and disciplines have different rules and procedures for releasing data - some do it as a matter of course, some do it on request. What constitutes "being released for general review" is therefore a matter of interpretation, to some extent.

That said, here's a list of fairly prominent examples where AGW data and models have been released.

I've already linked to the National Research Council's report vindicating Mann , which is sort of the ur-controversy. Said report caused the release of all relevant data, and is as thorough an investigation into this kind of disagreement as we are ever likely to see.

Gavin Schmidt, a climate researcher at NASA's Goddard Institute and co-founder of the Real Climate blog has data and code archives, including the GISS E model here. A somewhat earlier, more user-friendly model is "here": http://edgcm.columbia.edu/software2/edgcm-climate-modeling/

The model data from the 4th Assessment Report of the IPCC is here

The National Center for Atmospheric research CCSM model is here

Canadian Center for Climate Modeling has several versions of their models here

NOAA's Geophysical Fluid Dynamics Lab's CM2.x model is here

The University of Victoria's climate model is here

Even the infamous CRU has a substantial amount of data archived

Note also that many of the model links above have links to resulting data and papers in close proximity.

If I'm wrong, and the code and data has been released, I'll be happy to post a picture of a large crow as the banner on this site for a month.

I'll believe it when I see it, AL.

Kinda sounds like Nixon's White House claiming they release all kinds of tapes all the time. Why be hung up on this one specific tape, there are tons of tapes out there.

Not convincing.

Jeff, there've been plenty of fallacies in your arguments and on your side of the aisle as well. Cut to the chase - if you've read the IPCC, then cite the bits you found "vastly wanting as science". Hell, directly address any relevant scientific papers, research, or data that you disagree with, and say why you do.

Because until then, you're doing a great job of acting like you're on board with the scientific process without actually getting your hands dirty dealing with the factual arguments under discussion. And I find that as worthy of at least as much contempt as you've thus far demonstrated towards me.

Chris,

I am perfectly happy to leave the code review and modernisation of the climate up to the agencies involved, a task that I understand has been undertaken but not completed. I'm am also willing to take Freeman Dyson at his word that they are reasonably good at computing the large scale hydrodynamics of the atmosphere, and I also am willing to take his opinion that the interactions with the oceans, currents, and clouds is not that well understood and requires real field work and not just modeling. And I think any reasonable person would then want the models verified against reality, something that isn't that easy given that suitable measurements are a recent innovation. And given the failure of the models to predict temperatures in the last decade it isn't unreasonable to regard them as incomplete and unverified at this point. No doubt some feedback mechanisms have been incorrectly accounted for, but since it is feedback from these parts of the models that makes the difference between catastrophic warming and the sort of rough equilibrium that seems have actually been the case over the last several hundred million years, it isn't just being difficult to question whether or not the science is settled

But these things are not what is at issue here. McIntyre was not able to get the code and data from Mann, he had to luck out and find bits accidentally exposed on the net. And then he found an elementary error which gives me no confidence whatsoever in the analytical abilities of Mann et al. What the emails expose is an attitude towards science and review that is incompatible with making sound public policy, or science, for that matter. That these people have such a large role in determining that policy and driving the agenda should give anyone pause.

I think everyone here agrees that people have an effect on climate. What is at issue is how much, and that has not yet been determined to a satisfactory degree. Certainly not to the degree required to justify draconian measures that should only be considered in the face of eminent disaster or extinction. But such measures do have an appeal to those seeking political power and that is another reason to require the utmost transparency in those parts of the data that seem to have an undo influence in the decision process. In this sort of thing there is no excuse whatsoever for keeping the data sequestered. We aren't talking about small university groups doing their thing, we are talking about publicly funded research on which highly intrusive regulations are going to be based. It isn't business as usual no matter how low you set the bar for "usual".

Mike’s Nature Trick (link): making the graph tell the story you want it to tell, by hook or by crook.

The Blogprof is all over these emails (link).

Chris - I'd read the NRC's report on Mann, and my intial report was that it 'felt' kinda self-referential - someone needs to do a network map of the papers it cites in support.

But rereading it, nowhere in the link you present can I find Mann's underlying data or computer models (there's an example model, but it's not Mann's core model)...so if those can be found anywhere, I'd love to be pointed at them

Marc

AL, McIntyre's blog has an entry from '05 which states the data has been released on an FTP server which I can't currently access; from what I can tell, this is a mirror of the relevant data and code (specifically the multiproxy.f file in the METHODS directory.)

That said, clearly the NRC report, while not containing the data itself (although the investigatory process which produced it did force the release of Mann's complete data, over the objections of bodies such as the National Science Foundation) does investigate Mann's research deeply, and largely discredit's McIntyre's idea that the "hockey stick" is merely a statistical artifact, from what I can tell.

Marc,

someone needs to do a network map of the papers it cites

That reminds me of two groups who were doing much the same thing: determining diatomic bonding potentials using the RKR method. As it happened, they used different numerical methods to invert the integral equation and the citations by each group completely ignored the other group. I've always wondered if that was by design, or if they simply didn't read the others' papers. Science can be surprisingly insular.

Phil Jones emails Steve [Schneider], editor of Climatic Change [plus others, editorial board of the journal?], telling him he shouldn't accede to McIntyre's request for Mann's computer code. In later email to Mann ("For your eyes only, delete after reading") Jones says he told Jones separately [presumably meaning without saying to the rest of the board] that he should seek advice elsewhere and also consult the publisher and take legal advice.(1074277559)

Doesn't sound like open science to me.

from what I can tell.

The Wegman report invalidated it, they couldn't reproduce it. But most convincingly, it didn't happen. Even an email refers to that "embarrassment". You can't have a stronger invalidation of a scientific model than disagreement with the facts.

Anthropogenic global warming science should be done over, from scratch, gathering new data and building models from a blank page, by different scientists, ones not associated with the cabal of conspirators revealed by these emails.

It needs to be done with 100% transparency, enforced both legally and practically, with teeth. There should be no nondisclosure agreements of any kind, and anyone who tries to build another rat's nest like the one that has just been revealed should go to jail.

The disproportion between the vast costs of making policy based on the "mystery meat" of anthropogenic global warming science and the comparative pennies saved by giving the same old tainted stew a new stir is so great that this is justified.

And in the meantime, don't make policy based on anthropogenic global warming science.

Anthropogenic global warming science is a crooked house.

Clear it down to bedrock, pour new foundations and build a good house.

Chris, I can't tell if you're defending the actions of the CRU researchers on the basis that the collective body of research is accurate; or if you're defending the accuracy of the collective body of research on the basis that the CRU researchers' actions are unobjectionable.

Or... something.

Please clarify.

Please clarify also, while you're at it, if a normal part of the scientific process does include attempts on the parts of individual researchers to consciously "punish" editors and journals for publishing opposing research, and, separately, if a normal part of the scientific process should include it.

Chris, I'm digging into your claim that McIntyre got Mann's original source code and data, so will withold comment until I know more.

But I'd love to get your take on the source code comments highlighted on McIntyre's site here

; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions ; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.

Marc

There was the hockey stick controversy , since discredited by a National Research Council report, among other reviews.
Care to point out where in that report it was discredited? What was discredited was Mann's use of principal component analysis. See the The Wegman Report

Armed Liberal, I believe McIntyre did eventually manage to get the code, although it took years and intervention from the US Congress to do so. By that time he had, with great difficulty, already pieced together an "emulation" which was close enough and achieved roughly the same result. He used that emulation to discover many of the flaws in the algorithm.

Here are two PDFs which should help bring someone who hasn't been following the story up to date with the McIntyre & McKitrick criticisms. This includes their explanation of why they believe Mann response does not explain away their criticisms.

PDF 1
PDF 2

Perhaps you missed the part in the NAS report.
"While “strip-bark” samples should be avoided for temperature reconstructions, attention should also be paid to the confounding effects of anthropogenic nitrogen deposition (Vitousek et al. 1997), since the nutrient conditions of the soil determine wood growth response to increased atmospheric CO2 (Kostiainen et al. 2004).
The irony is the NAS panel didn't do their homework. In fig S-1 all 4 of the tree ring reconstructions that allegedly support Mann's conclusions also use the "strip-bark" trees. So much for peer review of the NAS panels document. Chris wrote:
As for the Wegman report, it itself was never formally peer reviewed (unlike the NRC report).
IOW, you can't find anything wrong with the Wegman report. Neither did the NAS panel as North testified:
CHAIRMAN BARTON. I understand that. It looks like my time is expired, so I want to ask one more question. Dr. North, do you dispute the conclusions or the methodology of Dr. Wegman's report? DR. NORTH. No, we don't. We don't disagree with their criticism. In fact, pretty much the same thing is said in our report. But again, just because the claims are made, doesn't mean they are false.
CHAIRMAN BARTON. I understand that you can have the right conclusion and that it not be–
DR. NORTH. It happens all the time in science.
CHAIRMAN BARTON. Yes, and not be substantiated by what you purport to be the facts but have we established–we know that Dr. Wegman has said that Dr. Mann's methodology is incorrect. Do you agree with that? I mean, it doesn't mean Dr. Mann's conclusions are wrong, but we can stipulate now that we have–and if you want to ask your statistician expert from North Carolina that Dr. Mann's methodology cannot be documented and cannot be verified by independent review.
DR. NORTH. Do you mind if he speaks?
CHAIRMAN BARTON. Yes, if he would like to come to the microphone.
MR. BLOOMFIELD. Thank you. Yes, Peter Bloomfield. Our committee reviewed the methodology used by Dr. Mann and his coworkers and we felt that some of the choices they made were inappropriate. We had much the same misgivings about his work that was documented at much greater length by Dr. Wegman.

If you actually go through the NAS report and reconcile McIntyre's points you will find his points were accepted.

Some folks have started taking apart the HARRY_READ_ME.txt file I mentioned earlier. See here though you'll have to wade through a fair amount of digressions. (No, I have no idea why this is happening on what appears to be a stock market bulletin board.)

I skimmed portions of this Friday evening, before prior weekend commitments came due. I have to say that 'Harry' has my sympathy. A couple of ages ago I had the pleasure of figuring out how to merge historical meteorological data, complete with missing values, miscodes, and magically appearing/disappearing/moving stations, with some of the very first satellite based field sensors. Wasn't a lot of fun. And unlike Harry, I was not trying to replicate a previous set of results where the intermediate data had been trashed, the input data was global in scope, and many of the previous programming steps were undocumented or just plain gone. Whoever the 'Tim' mentioned in the file is (maybe someone who's followed the plot more closely knows) he has a lot to answer for (and may just get the opportunity).

Anyone aware of modern software engineering, code management, and data archiving practices is going to cringe reading this stuff. It's just incredible that it's being used as the basis for global nannyism and potentially trillions in graft.

It's sort of fascinating how many people on this thread are eager to debate me, but haven't actually bothered to read what I've written - in some cases, even when I was directly and specifically addressing their arguments.

Greg F, I already answered your earlier comment re: the Wegman report. Posting that comment over and over again will not change anything - the Wegman report is highly questionable, and regarded by many as a politically motivated hit piece, as even Rep. Whitfield admitted.

Chuck and NicolasV, I highly recommend you familiarize yourselves with the NRC report, or, if you already are, point out how it's actually flawed in your opinion. Because that delves far more deeply into the issues and science surrounding Mann's work than we're likely to get into here, and as it is, you're mostly just asserting that Mann's work was flawed without engaging with any of the existing arguments as to why that is or isn't so.

David Blue, I'm utterly unsurprised that you're now essentially arguing that all existing climate science work should be discarded (and the climate scientists involved presumably sacked, since they're not to be involved with the "new and improved" climate science work going forward). And, let's see, you're arguing that scientists who build up work that you consider a "rat's nest" should go to jail. Out of curiosity, who'll be the one deciding what is and isn't legal procedures for scientists to follow?

Care to explain again how this comment was "putting bad words in your mouth" again? I was merely implying you wanted to see political controls put in place to control scientific research - you're bending over backwards to directly state that's what you actually want to see.

And Armed Liberal, give it up. you were quite positive that

The basis of the claim on AGW is a set of computer models, which have never, to my knowledge been released to be peer-reviewed.

You asked me, point blank, where those models and data were. You said if I could provide pointers to said code,

I'll be happy to post a picture of a large crow as the banner on this site for a month.

Now you're engaged in silly games as to whether Mann satisfied McIntyre's every critique, but whether he did or didn't (and again, the NRC report comes down pretty strongly on Mann's side), that's simply beside the point.

You were wrong that the models had never been released to public scrutiny.

And the fact that you were wrong says a great deal about your level of ignorance regarding legitimate AGW debate, vs. what gets tossed around the echo chamber of conservative blogs like this one. Hell, I knew about the GISS E archive in advance of this comment thread, but it took me all of five minutes of googling to find most of the links I made to other climate models above. You're more than happy to slam climate science as fraudulent, but you apparently didn't even make a cursory effort to examine what climate scientists have actually provided, data-wise.

How all this ties in to my ongoing thesis that AL is basically willing to jump on any bandwagon that makes liberals look bad, regardless of the validity of the arguments involved, is left as an exercise for the reader.

Meanwhile, AL, put up or shut up regarding the model data.

Adios for now, kids.

There is at least one outstanding issue, the calculation of the error bars. One would presume that there was code to do this, Steve does not have that code. Another possibility is there is no code to calculate the error and the error bars were just made up.

Read # 85 Chris.
The NAS panel didn't call the Wegman report "highly questionable". In fact North said:
We don't disagree with their criticism. In fact, pretty much the same thing is said in our report.

I highly recommend you familiarize yourselves with the NRC report and reconcile it with the Wegman report since it is clear the NAS panel doesn't agree with you.

Here's what a real live climate scientist has to say:

"1. Transparency. Climate data needs to be publicly available and well documented. This includes metadata that explains how the data were treated and manipulated, what assumptions were made in assembling the data sets, and what data was omitted and why. This would seem to be an obvious and simple requirement, but the need for such transparency has only been voiced recently as the policy relevance of climate data has increased."
...
In summary, the problem seems to be that the circling of the wagons strategy developed by small groups of climate researchers in response to the politically motivated attacks against climate science are now being used against other climate researchers and the more technical blogs (e.g. Climateaudit, Lucia, etc).
...
But the broader issue is the need to increase the public credibility of climate science. This requires publicly available data and metadata, a rigorous peer review process, and responding to arguments raised by skeptics. The integrity of individual scientists that are in positions of responsibility (e.g. administrators at major research institutions, editorial boards, major committees, and assessments) is particularly important for the public credibility of climate science. The need for public credibility and transparency has dramatically increased in recent years as the policy relevance of climate research has increased. The climate research enterprise has not yet adapted to this need, and our institutions need to strategize to respond to this need.

link
Doctor Judy Curry, Georgia Institute of Technology Professor of Atmospheric Sciences

Chris read my comment #85. It is clear that the NAS panel didn't consider the Wegman report "highly questionable". Dr. North testified concerning the Wegman report:
We don't disagree with their criticism. In fact, pretty much the same thing is said in our report.
It appears Chris the only thing questionable is your interpetation of the NAS report.

Chris, #87:

Adios for now, kids.

I'm hurt.

Chris:

It's sort of fascinating how many people on this thread are eager to debate me, but haven't actually bothered to read what I've written - in some cases, even when I was directly and specifically addressing their arguments.

In principle, follow up discussion would be nice, since you did reply to some things, I said. But: (1) I felt those abstract issues were going off-topic. This is a discussion about a specific scientific conspiracy, with big consequences. (2) You have a habit of shifting ground with nearly every post. It doesn't do much for productive discussion, when you need to be called for making the same non-legitimate move over and over.

Chris:

And, let's see, you're arguing that scientists who build up work that you consider a "rat's nest" should go to jail. Out of curiosity, who'll be the one deciding what is and isn't legal procedures for scientists to follow?

There's two options, given that I'm saying that secret, private information has proved to be an unsound foundation for "science" on which gigantic public policy positions are based.

You can do this as a free project, open source, open everything, and free beer too. I think that gets you a climate wiki, created and maintained by unpaid volunteers. I don't think that works, but if someone can explain how it does work I'm all for it.

The other way is with lots of government money. In that case, legislators set the rules.

The rules have to set to prevent secrecy - to prevent the scientists on the project from again making themselves in effect the owners of priestly secrets, and punishing critics and competitors, as has been happening.

If you are taking the government money, you can't sabotage the openness which is essential to the trustworthiness and thus the value of the project.

Chris:

Care to explain again how this comment was "putting bad words in your mouth" again? I was merely implying you wanted to see political controls put in place to control scientific research - you're bending over backwards to directly state that's what you actually want to see.

No you weren't. You were trying to fit me up with a bunch of "yahoo" positions of your own invention.

That consistent bad faith about what you implied and what your opponent said, as opposed to what you tried to make him or her say, goes a long way to make you not much fun to talk to.

Greg F, you might want to work on your double posting issues. That said...

Chris read my comment #85. It is clear that the NAS panel didn't consider the Wegman report "highly questionable". Dr. North testified concerning the Wegman report:

We don't disagree with their criticism. In fact, pretty much the same thing is said in our report.

Actually, North also testified:

My comments today will highlight the findings of our
committee's recently released report. Its aim was to asses the
state of scientific efforts to reconstruct surface temperature
records for the Earth over the last few thousand years, and to
comment on the implications of these efforts for our understanding
of global climate change. Surface temperature reconstructions are
only one of many lines of evidence supporting the conclusion that
the climate is warming in response to human activities. These long
records give context and perspective to the issue but they are not
the primary evidence. In fact, human-induced climate change is
quite real.*

He goes on to say that they do have issues with some of Mann's statistical approaches, but those approaches do not fundamentally change the validity of Mann's conclusions regarding 20th century warming. Moreover, if you take a look at Dr. Crowley's testimony in the committee hearing, he does have some substantial disagreements with the Wegman report (not surprising, given that it wasn't actually peer reviewed:)

The conclusions and recommendations of the Wegman Report have some serious flaws. In addition to a number of technical errors, large and small, the following comments can be made in the bullets on page two of the committee's summary of findings (fact sheet): (a) bullet one (concerning specifics of Mann et al.) -
responses discussed above (b) bullet two - "many of the proxies are reused in most of the papers....it is not surprising that would obtain similar results..." This almost sounds as if it is wrong for everyone to use the best existing data! The more important point, and one not stated, is that different methodologies are employed by each of the
investigators. Furthermore, there is nothing wrong with talking to or even collaborating with someone else in a field that you respect, and has expertise that you don't have. The Wegman Report almost seems to imply that collaboration is equivalent to collusion, a result that would apply to the Wegman Report itself if that were always true.

...

In my view the debate over the Mann et al paper is a tempest in a teapot. It is legitimate material for scientific discussion but the implications with respect to the operations of the IPCC are unproven and seemingly based, in my opinion, much more on repetition of innuendo than on any real facts. Although there is always a need for enhanced interaction with the statistics community, the lack of communication is seriously misrepresented in the Wegman Reprot. I believe that this report should not be used as either a legitimate assessment of the science or as a guide to policy modification.

So yeah, my interpretation of the report is pretty much in line with what was actually said at the committee, Greg. You, on the other hand, might want to reexamine your one cherry-picked line of testimony that you're trying to use to imply that North and the NRC report actually disagree with Mann's conclusions, when the opposite is true.

Bullet (b) above is the problematic one, though.

Even if you say "they're using the best available data," it's extremely important to remember that the data is not particularly good. Specifically, it's not that well-correlated with actual temperatures. It's not that the researchers are being malicious in that respect - we can't use time machines to go back and measure this stuff, after all.

But it's very important to pay attention to the proxy data, because it's already not very good to begin with. Where the proxy data diverges from recorded temperatures, you need to have noted it, because if there are confounding factors involved (and, obviously, there are), then that affects your model's inputs more or less across the board.

What these e-mails caught the researchers doing was noting that the proxy data gave weird results when plugged into the model, at variance with temperature records we do have, and changed the inputs to make their output look like the historical record. Bad, bad, bad scientific practice. Taking data of already-dubious quality and adjusting it because otherwise your model will throw out results that clearly aren't true... What does that say about the accuracy of your model, which is, after all, only a gross approximation of physical processes that we don't understand and can't model in the micro sense?

Even that could be chalked up to sausage-making, if it wasn't paired with instructions to delete e-mails. That's the damning part. You don't instruct people to delete e-mails on a topic (with FOI in the subject of the e-mail!) unless the conversation you've been having is extremely damning, worse to your case than reading that you're defying orders to produce the information and destroying it instead.

Furthermore, I'll quote a quote from earlier:

I know about what Matthews has done. He did so without contacting Sarah or me. He uses a statistical emulation method that can never account for the full range of uncertainties. I would not trust it outside the calibration zone -- so I doubt that it can work well for (e.g.) stabilization cases. As far as I know it has not been peer reviewed. Furthermore, unless he has illegally got hold of the TAR version of the model, what he has done can only be an emulation of the SAR version.

There is no universe in which someone can write this e-mail about a statistical model which has had its code and methodology released to the public.

"He goes on to say that they do have issues with some of Mann's statistical approaches, but those approaches do not fundamentally change the validity of Mann's conclusions regarding 20th century warming."

In other words, so long as your conclusions 'seem' valid, you can do whatever you like to get there.

Science!

Chris,

I want to thank you for your efforts here and the time and work you put into your presentations. It certainly stirred a pot on a number of issues that should have been addressed on the site.

I would like to correct you on one point, though, you have referred may times to people's Conservative views and Conservative talking points.

In many cases, you should have used the term Reactionary.

Uh huh.

Anyway- i'll play the crystal ball game a little here and suggest that (demonstrably) this story will be swept under the rug or stonewalled by the majority of the media. There is good possibility of criminal wrong doing here (regarding FOIA), but its in everybodys best interest to ignore it which generally means it goes away.

On the bright side, it seems that the scandal may encourage dissenting voices in the climate research industry to speak up, and perhaps prevent people like Jones and Mann from keeping them from being published. And i'm not talking purely about 'skeptics', but about honest questions being asked by fellow researchers regarding specific studies and data.

The red herring here is that skeptics are out to 'debunk' the warming of the earth. A few are, but most are concerned with the lock step political agenda that requires the earth to be: A.warming B.because of CO2 C. at a rate that is catastrophic to mankind. If C is not true, A&B become much less relevant. But this clique seems to have an agenda to make sure C goes unquestioned, when in fact it is the most questionable. If the data suggests the warming of the earth is neither unparalleled nor imminently catastrophic, the political impetus withers. What is disturbing is that scientists are pushing that particular agenda. The red herring is suggesting they are manufacturing warming where there is none. It is more than enough to exaggerate the current warming while covering up warming in other eras.

toc3, forgive me as I laugh my rear off. We haven't discussed policy at all; we're making an effort to judge the quality of science on which we're being asked to judge policy - and having a very preliminary take at it.

If looking at c**p like that released and calling it bad science is Reactionary, I'm a little worried about what it means to be Progressive.

And that's coming from the guy who drives a hybrid, has PV solar on his roof, and suggested that we could meaningfully impact our energy budget by dropping SUV's and buying minivans or crossovers.

Marc

Welcome to the reactionaries, Marc. Every thinking person is eventually hauled off by the progressive enforcers and tossed in here with the rest of us. Resistance is futile, your puny PV panels mean nothing. Embrace the suck.

Avatar-

Even if you say "they're using the best available data," it's extremely important to remember that the data is not particularly good. Specifically, it's not that well-correlated with actual temperatures. It's not that the researchers are being malicious in that respect - we can't use time machines to go back and measure this stuff, after all.

Do you have a cite for the idea that the proxy data "isn't particularly good?" Preferebly one that doesn't have it's origins from the McIntyre and McItrick side of the fence? I haven't dived very deeply into the science on proxy data, but what I've read does suggest that there are good reasons to believe that ice core data, tree ring data, sediments, etc., are all actually pretty good proxies for historical climate records.

As for much of the rest of your accusations about tampering with data, etc., I'm extremely skeptical that such stuff isn't being taken out of context, and I'm more than willing to wait for any relevant scientific review processes to kick in. Just as WoC denizens are skeptical as hell of climate scientists in general, I'm equally skeptical of the spin that folks like AL - who have a well-established antipathy towards climate science but, as events have shown, little understanding of the actual data that's available and the level of skepticism this work has already encountered - will give to these emails.

That said, there is one bit that's worth addressing here:

There is no universe in which someone can write this e-mail about a statistical model which has had its code and methodology released to the public.

Even if we take that as a given, Avatar, there are legitimate reasons why models and data might not yet have been released (legal agreements with funding agencies, IP issues, etc.) That doesn't mean that the data never were and never will be released (note that the timestamp on said email is ~2003).

And whatever the issue is with the model in question, there have been plenty of other models released. In fact, assuming the SAR and TAR models under discussion refer to the Second and Third IPCC Assesment Reports, the model data for the Fourth Assessment Report has already been posted and linked to above.

Whatever flaws there may or may not be in the CRU, implying that all climate science is equally flawed isn't just irrational, it actively flies in the face of all the other publically released science that I've linked to throughout this thread.

Chris, there's a world of difference between citing papers and understanding (or being able to explain) science.

So here's a proposal for you. Why don't you propose what you believe to be the three or four foundational papers for AGW science, and let's - over a period of time - go through them together. Maybe we vast unwashed unlearned ignoramuses can learn something. I'm happy to offer you several posts at Winds to lay them out.

Maybe we can move this to a constructive discussion and away from the "pound sand" that we're rapidly approaching.

Marc

Thanks to Chris, I've got a ton of reading ahead of me for the weekend. But one small thing popped out in Chris' comment #95:

Furthermore, there is nothing wrong with talking to or even collaborating with someone else in a field that you respect, and has expertise that you don't have. The Wegman Report almost seems to imply that collaboration is equivalent to collusion, a result that would apply to the Wegman Report itself if that were always true.

Isn't that collusion - attempting to remove people from editorial positions, denying information to people who don't support one's views, working to ensure that opposing papers weren't published,and that supportive ones were - EXACTLY what's in view in the emails we're talking about?

Marc

If you guys want to play Real Climate Cowboys vs. Climate Audit Indians, I wish you the best. At least that debate doesn't cost me money.

To me the story of these emails is that a bunch of government-fed scientists have set themselves up as a priesthood, with a self-justifying secret scripture that only they are allowed to interpret, and have set out to personally destroy the heretics that question their authority - an authority which now extends way the hell out of their academic field.

The list of people that they want to personally destroy is probably far longer than the lengthy index in these emails, and include people like Hans Von Storch, an AGW proponent who had the temerity to question their questionable methods.

And yet these people are not one tenth of the problem. The real problem are the laymen who believe everything these climate Jesuits say and defend everything they do, because they see the endless opportunities for political exploitation.

Enjoy it while it lasts, boys and girls. Even the communists eventually wised up to Lysenko.

Its all about transparency. If you are working on freely available data in a fair review and publishing atmosphere, there is nothing wrong with collaborating. Everything is visible to any interested party.

On the other hand if data and methods are hidden and dissent is blocked and demonized, you get... this.

On a lighter note, perhaps the CRU modelers could buy some bad code offsets in penance for their sins.

Forgive me Mark if I wasn't clear as to what I meant.

First, Chris put up a very spirited and reasoned defense of his argument against a host of adversaries, stirred a lot of reaction, commentary and fleshing out of people's position. I noticed you haved been sufficiently challenged to read some of his citations. Something I would think you would appreciate.

Secondly, before the election, you might remember my constant defense of conservatism against the Neo-Cons and the more reactionary wing of the Republican Party.

It now appears that not only not only Liberals are demons, but RINOS have joined them in the eyes of the Reactionaries.

So, let's call a spade a spade. Much of the commentary was reaction. As is a lot of the other commentary on the site. In many cases, this reactions is anti-conservative.

You can see that

1. In town meeting with people like Lindsay Graham, of all people, who was excoriated at a recent meeting with his constituency for looking for any sort of compromise with the Democrats.

2. A brainless embrace of people like Beck and a subservience, which to his everlasting credit Graham spoke out against, to media morons like the above mentioned Beck and Rush Limbaugh.

3. A general shrinkage of the Party into a regional southern base with a religiously based social agenda.

This is not the Conservatism of Goldwater, nor Reagan nor, the new whipping boy of these "Crypto" Dixiecrats, Newt Gingrich, it is now become a reaction to it

So, I was just politely correcting Chris. Conservatism is not Reaction. Never has been, Never will be.

toc3, what the heck is "Reaction"?

Whether AGW is "true" or not is irrelevant. A crooked scientist is still a crook, even if his theory stands up.

Did you post his for any particular reason, or was it that you thought you would somehow be scoring points by doing so?

No, I honestly don't know what you mean by "Reaction", although you appear to think the concept is important. And yes, I know what a "reactionary" is, and don't see how it applies on a discussion of scientific fraud.

If you defined it elsewhere, a link would be useful.

I think its been clear for a long time that toc has his own definition of conservatism that no-one else on earth seems to understand, much less conform to. Which I think is just fine with everyone else on earth.

Did you post his for any particular reason, or was it that you thought you would somehow be scoring points by doing so?

Do you have a cite for the idea that the proxy data "isn't particularly good?" Preferably one that doesn't have its origins from the McIntyre and McItrick side of the fence?

You're asking for a cite to the idea that the proxy data is flawed, and it must come from the people who maintain it isn't?

Reaction, in the sense I was using to describe the emotional way Reactionaries respond to ideas that threaten them or their political belief system. Chris came up with some points that were provocative. He was met by some intelligent conservative views and a lot of reactionary nonsense. The same type of mindless point scoring I thought you were engaging in.

An example of mindless point scoring is Post 111 in this thread.

I apologize for mistaking you question for this type of post.

My original post was to compliment Chris for starting an intelligent and rigorous debate which is what I come here for, which led me to also comment on how the Reactionaries detracted from the debate with most of their commentary.

My second was to reply to Mark and further clarify my points.

That just begs the question of what a Reactionary is, though. You still haven't defined anything, and you've given only one example. Or does it just mean anything you disagree with?

Excuse me for interrupting this fascinating discussion on the meaning of "reactionary", but I found this short code review amusing. And the link to thedailywtf provides some entertainment for the geek.

Well, if the definition of something is being begged for, by you in this case, I would think the mendicant would be wise to look it up.

Or are you begging me to look it up for you? If that is the case, the answer is... No.

Liberals, Democrats and Radicals are not Reactionaries and I don't agree with them, so the answer to your question is also no.

Excellent Post

Great site. Maybe we should move the debate over there. Much more exciting and with it venue. :)

As for the code review, I've seen a ton of awful FORTRAN when I was designing death for bux at the missile factory :) There is always a tension between The Scientists, who tend to be pathetic coders, and the programmers who don't understand the theory. And in many cases - ie, probably this one - academic environments don't have professional programmers.

In well-done scientific programming, you have developers write a framework and APIs and let the scientists implement The Magic Algorithm in an environment where they don't have to do things like file I/O, sorting, etc.

One of my favorite bits of awful "scientific programming" in FORTRAN was a bit of sort code where they set an array full of "max vals" and then passed the entire 10M element array to the sort code. After I changed the code to only sort the used part of the array - which was usually far, far less than half - the sim ran in a half hour instead of two weeks.

Putting in heapsort instead of "it's shorter - doesn't that mean it's more efficient?" bubblesort helped too.

This, as much as anything, is why I'm hugely suspicious of magic sims that we're supposed to trust sight-unseen as the basis of trillion-dollar decisions.

It's just that you are using the word in a way that makes no sense to me. Clearly, you have something in mind. I am curious what it is. But feel free to drop it, as it is a fairly meaningless sideline. In fact, I only brought it up because someone else already had, and your response was... less than useful.

But like I said, no big deal, and no reason to pursue it.

1. The NAS panel stated that strip bark pine should not be used for temperature reconstructions.

2. Apparently unknown to the NAS panel at the time, all 4 of the reconstructions from tree rings the NAS panel cited use the strip bark pines.

3. Peer review is not a magic bullet. The fact that the NAS panel used reconstructions that, contrary to their recommendation of not using strip bark pines. This makes a mockery of your peer review appeal to authority.

4. When asked for specific points about the Wegman that were wrong you can't answer. All you can do is repeat the 'it wasn't peer reviewed mantra' as if nothing is true if it isn't peer reviewed. IOW, you are unwilling or unable to address the issues the Wegman report raised. Attacking the messanger is not science. It is faulty logic. You also point to Crowley's statement, who is hardly a disinterested party. Does Crowley call into question Wegman's analysis of Mann's statistics? NO! He complains about the social network. He complains about the "best data" which includes the strip bark pines. The question should not be using the best data. It should be is the data good enough?

5. The NAS panel was not peer reviewed, they were the peer review. Wegman was also peer review on Mann's methods.

6. North's statement that you quote was prior to the pointed question where he admitted they agreed with the Wegman report.

In comment #101 you ask:
Do you have a cite for the idea that the proxy data "isn't particularly good?"
Are you familiar with the logical fallacy of proving a negative? Here is one for you Chris. How do you seperate all the factors that contribute to tree ring growth? A not entirely inclusive list would include.
1. Water avalability
2. Soil
3. Temperature
4. Age
5. Shading
6. Competition from other vegitation
7. Insects

So how do you solve an equation that is non-linear with at least 7 variables that you don't have any resonable data for?

Putting in heapsort instead of "it's shorter - doesn't that mean it's more efficient?" bubblesort helped too.

Reminds me of the super wonderful interpolation program that used linear search to find the interpolation interval. In the usual case of interpolating at large sets of ordered points it was O(m*n^2), where m was on the order of thousands and n about 500. People wondered why its advertised "optimized" performance wasn't better.

I suspect the introduction to programming of most aging scientists, meaning boomers like myself, was rather informal. Mine surely was, the guy I was helping in the lab handed me a Fortran manual and a technical report on the Gauss-Newton method, and sent me off to do a non-linear least squares fit. That was back around 1967 when I was a student.

More goodies:

ARGH. Just went back to check on synthetic production. Apparently - I have no memory of this at all - we're not doing observed rain days! It's all synthetic from 1990 onwards. So I'm going to need conditionals in the update program to handle that. And separate gridding before 1989. And what TF happens to station counts?
OH FUCK THIS. It's Sunday evening, I've worked all weekend, and just when I thought it was done I'm hitting yet another problem that's based on the hopeless state of our databases. There is no uniform data integrity, it's just a catalogue of issues that continues to grow as they're found.

Harry is my prime suspect for the leaker. He got saddled with cleaning up the mess and I'll bet he didn't get much help.

Sad and funny at the same time This is the software which the global temperature record is based on.

Poor Harry trying to sort out the programming mess that generates the CRU instrument record.

Nothing to worry about. I am sure it has all been peer reviewed!

Greg F-

The NAS panel stated that strip bark pine should not be used for temperature reconstructions. Apparently unknown to the NAS panel at the time, all 4 of the reconstructions from tree rings the NAS panel cited use the strip bark pines. Peer review is not a magic bullet. The fact that the NAS panel used reconstructions that, contrary to their recommendation of not using strip bark pines. This makes a mockery of your peer review appeal to authority.

Greg, you're referencing one sentence buried deep in the NRC report that basically says "strip-bark samples should be avoided for temperature reconstructions." (Note that "strip-bark" is actually a state of growth that older bristlecone pines sometimes enter, and not a specific species of tree.) Insofar as strip-bark samples are problematic, the question is how many of them were used, and did those strip-bark invalidate the resulting data. If you can show links to proof that those samples were used and did invalidate the data, go nuts, but that doesn't change the fact that the NRC report spends a great deal more time talking about the value and utility of tree ring data, and how the tree ring data matches with other, wholly independent proxy data which shows substantially the same results as the tree ring data.

This is not an appeal to authority, this is reasoned science, laid out in great detail in the NRC report.

That said, there's been at least some recent research which indicates that strip-bark samples are not significantly different than non-strip-bark bristlecones, and would thus be acceptable for paleoclimate reconstructions.

When asked for specific points about the Wegman that were wrong you can't answer. All you can do is repeat the 'it wasn't peer reviewed mantra' as if nothing is true if it isn't peer reviewed.

This is simply not true, based on your own recognition that I referenced Crowley's remarks. And Crowley has five separate critiques of the Wegman report, at least one of which has to do with Wegman's analysis of the statistical methods employed by climate science in general. Perhaps Crowley does have biases with regard to defending Mann, but I suspect he's ultimately a more unbiased party than the Republican congressmen responsible for kicking this thing off in the first place.

The NAS panel was not peer reviewed, they were the peer review. Wegman was also peer review on Mann's methods.

The NRC report was, to the best of my knowledge, extensively reviewed by uninvolved third parties before publication. Wegman basically sent his report around to his close friends. Not really the same standards, Greg.

North's statement that you quote was prior to the pointed question where he admitted they agreed with the Wegman report.

That doesn't invalidate what he was saying about the Mann's conclusions being fundamentally valid.

Are you familiar with the logical fallacy of proving a negative? Here is one for you Chris. How do you seperate all the factors that contribute to tree ring growth?

Greg, you don't have to prove a negative, you simply have to show that the data is essentially not valid by debunking the many references in the NRC which extensively cover why tree ring growth is believed to be an accurate and useful proxy for historical climate data.

Good luck with that.

You know what would be really suggestive of real GW? If someone would plot all the different proxy series (raw, uncorrected data) over time against each other, and against known, measured, uncorrected temperature data from when that is available. If those series matched in the observed range, that would be good evidence that they were trustworthy as proxies in older time periods.

Off I go to bed, but tomorrow I think I'll look to see if this exists somewhere. I'm not hopeful, because I suspect that "we don't have the data" means among other things that that hasn't been done and now cannot be done. But I'm going to look, because it would be such a beautiful result if it does exist.

Still wouldn't prove AGW, but at least it would be a step forward past the "is earth warming, and over what time frame" argument.

Uh oh, an overflow problem or something...

17. Inserted debug statements into anomdtb.f90, discovered that a sum-of-squared variable is becoming very, very negative! Key...

I'll bet they are summing up squared integers somewhere. Or maybe the numbers were imaginary all along. But in any case I think we may have stumbled on the key to recently decreasing temperatures.

If those series matched in the observed range, that would be good evidence that they were trustworthy as proxies in older time periods.

They don't, that's why the trees used have to be selected. It's basically all crap, as I think we will soon discover.

Chris, there's a world of difference between citing papers and understanding (or being able to explain) science.

Yes, and I didn't say otherwise, AL. Moreover, I never claimed that this was any kind of contest over who could explain the science the best, or even who understood AGW science, or the scientific process the best. I claimed that this was a situation where your opposition to AGW science was irrational and uninformed. And I think I proved that.

So here's a proposal for you. Why don't you propose what you believe to be the three or four foundational papers for AGW science, and let's - over a period of time - go through them together. Maybe we vast unwashed unlearned ignoramuses can learn something. I'm happy to offer you several posts at Winds to lay them out.

AL, I respect and appreciate the offer. That said, I'm going to decline for two reasons:

1. I don't believe the vast majority of WoC readers are willing to approach such a symposium with a fair and open mind. Even if they did, I'm fairly certain they would trust me to present a true and logical account of AGW theory, given that WoC mainstays like Glen Wishard have decided I'm "beneath contempt."

2. I freely admit I'm not qualified to thoroughly explain the science and methodology behind AGW - I have degrees in tangentially fields, I know enough physics and mathematics to understand the basics of AGW, and I'm familiar enough with scientific methodology to be able to follow the broad outline of argument between climate scientists and skeptics. (And to not be terribly impressed with what I'm seeing on the skeptic side.)

But there's a reason climate science is carried out by dedicated professionals with years of specific training, and, contrary to what many here have suggested, it's not to keep that scientific knowledge as some kind of "priestly secret." Rather, it's because it takes a substantial amount of time and discipline to truly understand the physics, math, chemistry, biology, and computer science that climatology touches on, and to become familiar with dozens or even hundreds of relevant papers in the field. It is unquestionably a full-time job that requires a dedicated, intelligent person to fill.

You said it yourself - there's a difference between citing papers and being able to understand and explain the science. There's a reason we want college-level courses taught by people with years of relevant experience - senior grad students at least, if not full PhDs. And although those PhDs can teach advanced science topics to reasonably bright teenagers, there's also a reason why it takes weeks and months to cover and to truly understand a subject like basic kinetic physics, or data structures. One of the (many) unfortunate side-effects of the revolt against supposed "elites" on the conservative side is that legions of Glenn Reynolds wannabes think that the ability to use Google and skim a few search results makes them as knowledgeable their opinions as valid as as people who've spent their entire adult lives mastering that branch of scientific knowledge.

So go to somebody actually qualified to teach this stuff, AL, if you're honestly interested in learning it. Alternatively, if you want to do more than "pound sand," why don't you merely try debating this with the other people on this site?

Look, in this comment you agreed that there were "multiple, independent sources of data that support the concept of GW"; Jeff Medcalf says that he's "never been convinced that the earth is warming overall, though I'm willing to be if someone can produce good evidence." You might try seeing how far reasonable exploration of the science gets you with other folks on this site who disagree with you, and see how far that gets you, rather than trying to prove that you can out-debate me in science if you just keep at it long enough.

Isn't that collusion - attempting to remove people from editorial positions, denying information to people who don't support one's views, working to ensure that opposing papers weren't published,and that supportive ones were - EXACTLY what's in view in the emails we're talking about?

No, because what we're talking about is actually:

- attempting to remove people from editorial positions who are exhibiting unprofessional behavior as determined by a substantial number of professional peers,

- denying information to people who give every appearance of being engaged in a bad-faith effort to discredit your work for political reasons, and

- working to ensure papers that conform to accepted norms of scientific quality are published, and those that are not are not.

Or, rather, if more than that is to be believed, you'll need more evidence than a set of out-of-context, maliciously selected, stolen emails to back it up. (You do realize that the hackers in question have only released sporadic emails, rather than the entire stolen archive?)

I think that's all I've got for now, so I'll merely close by recommending John Timmer's excellent article on Ars Technica, which is even-handed and knowledgeable about the scientific domain.

And when does that crow banner go up, AL?

Jeff Medcalf-

You know what would be really suggestive of real GW? If someone would plot all the different proxy series (raw, uncorrected data) over time against each other, and against known, measured, uncorrected temperature data from when that is available. If those series matched in the observed range, that would be good evidence that they were trustworthy as proxies in older time periods.

Off I go to bed, but tomorrow I think I'll look to see if this exists somewhere. I'm not hopeful, because I suspect that "we don't have the data" means among other things that that hasn't been done and now cannot be done. But I'm going to look, because it would be such a beautiful result if it does exist.

chuck-

They don't, that's why the trees used have to be selected. It's basically all crap, as I think we will soon discover.

Right. The data doesn't exist.

This archive of tree ring data doesn't exist.

These charts plotting many different proxy reconstructions against the temperature record doesn't exist.

This report which is full of references to the original papers where that data was described, doesn't exist.

As Armed Liberal has proved, the relevant models aren't on the web, because they don't exist.

If you go to the websites of the labs and scientists who publish the papers cited in the NRC report, you won't find the data or instructions on how to get the data, because none of that exists.

This chapter of the report which describes in fairly specific detail why tree ring data proxies are useful, doesn't exist.

This bibliographic archive of research going back decades on exactly how to measure and compare tree ring data doesn't exist.

Likewise, this book which goes into great detail on how and why tree cores are selected (and they have to be "selected" just right for the conspiracy to work, right chuck?), doesn't exist.

Look at it all, not existing! The mind boggles!

And thank goodness it doesn't exist, because if it did - if you actually took a look at the data involved, and the preparations of the data that have been derived from decades of research by hundreds of full-time scientists on how and why said data needs to be prepared that way to be of use - then you'd realize that science involves more than finding a few tab-delineated data files that you can throw in Excel, make a few graphs of, and prove or disprove all of global warming on a spare weekend.

You know, Chris, that the lists of argumentative fallacies aren't actually todo lists, right? For example, the strawman argument ("you'd realize that science involves more than finding a few tab-delineated data files that you can throw in Excel, make a few graphs of, and prove or disprove all of global warming on a spare weekend") is not particularly convincing, especially when aimed at those whom you accuse of matching the strawman. (Look it up: if you want to use a strawman to win an argument, the point is to pitch it to the uncommitted audience, not your interlocutors.)

See, I finally realized that what is bothering me is that you are so close to being able to debate the issue reasonably. It's easier to deal with people who are utterly irrational (on both sides) than with someone who is apparently rational, but simply refuses to debate without throwing out so many fallacious arguments that it's a chore reading through their arguments.

You say: "I freely admit I'm not qualified to thoroughly explain the science and methodology behind AGW - I have degrees in tangentially fields, I know enough physics and mathematics to understand the basics of AGW, and I'm familiar enough with scientific methodology to be able to follow the broad outline of argument between climate scientists and skeptics."

And yet, earlier, you were utterly dismissive of the rest of us being in the same boat based on our lack of credentials which you yourself claimed: "I have a doctorate, I've worked in scientific academia, and these emails are very much in line with how people talk and deal with data. You, on the other hand, have no scientific background to make these claims that I'm aware of." Apparently, claimed rather falsely. (Not that you have a doctorate, but the implication that you have credentials to be taken seriously on this topic that the rest of us lack, in your estimation.)

Now, Briffa, Mann, et al are not going to drop in and educate us/answer our questions. You took on that role, a proxy if you will, by stepping into the debate on one side of the argument, just as we have in a sense taken on a role as proxy skeptics. You are doing a real dis-service to your arguments by making them so poorly, and with such high-handed dismissiveness, particularly when you then come clean and admit that you are not even able to stand on the (fallacious) claims you present! (Coming clean, if you missed it, is not the problem; making fallacious claims that you can't fulfill regardless is the problem.) I would really like to know the answer to whether the earth is warming, whether that is a problem or a benefit or even in any way out of the ordinary, and if it's a problem, what is causing it and how we might go about alleviating or palliating it. You aren't communicating helpfully towards reaching those answers, though I suspect you think you are. The data and studies you are linking to are useful, but at the same time in large part are exactly the data being undermined by these revelations. (I'm not talking about the emails, which simply show assholes at work, but at the models and data, which are pathetic in the extreme. Which is probably why they try to hide them, actually.) The convincing answer to "how can we trust the data" is not "here it is, and look at how much, and what pretty charts."

It's an important topic, and you seem like you are capable of generating more light than heat, but you're just not doing it, and that's why it bugs me so much reading your comments. Lost potential is more annoying than lack of potential. Oh, well.

Thanks for the link to the proxy data, by the way (though it looks like this is highly massaged data, rather than the raw original, but I haven't had time to look in detail yet). I'll dig into it later and see if I can't track down more from either the paper or its references.

The NRC report was NOT tasked with critiquing Mann's work. Wegman's was.

"The NRC put together a panel of twelve members including two statisticians (Doug Nychka and Peter Bloomfield) and others representing paleoclimate, climate modeling and general climate science. The charge to the committee, was “to summarize current information on the temperature records for the past millennium, describe the main areas of uncertainty and how significant they are, describe the principal methodologies used and any problems with these approaches, and explain how central is the debate over the paleoclimate record within the overall state of knowledge on global climate change.” This was deliberately framed to give a broader overview than the charge to the Wegman committee, which was specifically focused on Michael Mann's papers and the critiques of those papers.

The NRC's conclusions drew on the fact that Mann's results mirrored other evidence, which goes back to the house of cards mentality.

"A number of other commentators have acknowledged the flaws in the Mann reconstruction but have argued that this does not matter because the answers have been verified by other analyses. Ed's own response to that was given in the equation.
Method Wrong + Answer Correct = Bad Science.
In other words, the fact that the answer may have beencorrect does not justify the use of an incorrect method inthe first place."
ASA Section on Statistics and the Environment Newsletter, Spring 2007

All this just reinforces the mentality of the current crop of climate scientists that the most important aspect of their work isn't scientific methodology or accuracy, but that the results match up to the conventional wisdom. You either find that disturbing or not.

I guess this is Stage Two: George Monbiot calls for Phil Jones to resign. Says this is about "the credibility of three or four scientists."

Well, nobody cares about the credibility of three or four spotty scientists - at least, not after their credibility becomes indefensible. They can just be airbrushed out of the picture like dead cosmonauts ...

Again, excellent work.

I would like to speak to you about something I am working on. If I could send my Email to you through AL or vice-versa, I would appreciate it.

If you'll both send me email addresses, I'm happy to forward something. Otherwise, when you comment, I think you can add your email address to the metadata and it's viewable.

Marc

"Right. The data doesn't exist. "

Lets see what Phil Jones says about that:

If we have “lost” any data it is the following:

1. Station series for sites that in the 1980s we deemed then to be affected by either urban biases or by numerous site moves, that were either not correctable or not worth doing as there were other series in the region.

2. The original data for sites for which we made appropriate adjustments in the temperature data in the 1980s. We still have our adjusted data, of course, and these along with all other sites that didn’t need adjusting.

3. Since the 1980s as colleagues and National Meteorological Services (NMSs) have produced adjusted series for regions and or countries, then we replaced the data we had with the better series.

You're right! The data isn't "lost", its just been replaced by data for which Jones and co have made 'appropriate adjustments'. We've seen what the documentation for such adjustments are in the Harry meta-code.

This is apparently a new science that i'm not familiar with, wherein raw data should be replaced with adjusted data, apparently to avoid just this type of confusion as to what the 'correct' result is.

And Chris- i'll tell you again, you can't rebut criticism about missing data by citing all the other data that isn't missing. Try that with the IRS sometime- 'hey, you've got my returns for every other year, just not this year. Whats the problem?'

Isn't that collusion - attempting to remove people from editorial positions, denying information to people who don't support one's views, working to ensure that opposing papers weren't published,and that supportive ones were - EXACTLY what's in view in the emails we're talking about?

No, because what we're talking about is actually:

- attempting to remove people from editorial positions who are exhibiting unprofessional behavior as determined by a substantial number of professional peers,

- denying information to people who give every appearance of being engaged in a bad-faith effort to discredit your work for political reasons, and

- working to ensure papers that conform to accepted norms of scientific quality are published, and those that are not are not.

It takes an extreme act of mental double-jointedness to justify the active stifling of debate by these scientists by cloaking it in comfortable terms of process. This is beyond the pale and postiviley Orwellian; threats and intimidatation are what they are, and there's nothing usual or scientific about it. Your defense you perform these remarkable feats of intellectual dishonesty, apparently to your own satisfaction and with a straight face, tells me all I need to know.

Honest debate presupposes at least a passing adherence to available facts. Further debate with someone this blind to facts- papering over, denying or ignoring them- is both unwarranted and counterproductive.

Chris, with your post above, you put yourself outside legitimate debate.

Jeff-

You seem to have this odd idea of what rational debate should look like - that we are, or should be, in this high-minded Acropolis of pure knowledge, and that these debates should be scored by some kind of Aristotelian rules of proper conduct. ("Oh no! Strawman fallacy! Five points off for you, old chap!")

In fact, your obsession with identifying what you believe to be the parts of debate blinds to to what those parts are actually doing - the difference between knowing the notes of a song and actually playing it. Here's a example - yes, me saying the thing about the Excel spreadsheet is a strawman if you take it literally. It is not a strawman attack if you look at it in the wider context of you apparently being ignorant of the broader scientific process - e.g. seeking out the "raw," "uncorrected" data.

Here's the thing, Jeff:

A) the raw data is there, especially in the tree ring data repository

B) the raw data is raw tree ring data, and as such tells you nothing about temperature, in and of itself - it's just data about the widths of tree rings, etc. The data must be processed to be of any use in comparisons.

C) Figuring out how to correctly process the data is a non-trivial problem that it's taken climate scientists years to figure out, and really understanding how to do it, and why, and what constitutes valid, necessary processing from unethical and invalid processing takes years of study in an at least tangentially related field, as a rule.

So the idea that this argument could somehow be solved - or that it could even be significantly advanced - if you got your hands on the "raw, uncorrected data" is at fault. Which is what I was attacking in that passage, with a bit of hyperbole mixed in, which is in keeping with the sarcastic tone of the rest of the post. (And yes, sarcasm is a legitimate technique in debate - it's not nice, but it's legitimate.)

That said, one more thing I should point out.

You say: "I freely admit I'm not qualified to thoroughly explain the science and methodology behind AGW - I have degrees in tangentially fields, I know enough physics and mathematics to understand the basics of AGW, and I'm familiar enough with scientific methodology to be able to follow the broad outline of argument between climate scientists and skeptics."

And yet, earlier, you were utterly dismissive of the rest of us being in the same boat based on our lack of credentials which you yourself claimed: "I have a doctorate, I've worked in scientific academia, and these emails are very much in line with how people talk and deal with data. You, on the other hand, have no scientific background to make these claims that I'm aware of." Apparently, claimed rather falsely. (Not that you have a doctorate, but the implication that you have credentials to be taken seriously on this topic that the rest of us lack, in your estimation.)

Jeff, you're missing the context and the point of what I was getting at in those passages. In the latter, I was bringing up my scientific experience to talk about how scientists sometimes talk about data. In the former, I was discussing my qualifications to explain (what I'm guessing would have turned in to) the entirety of global warming. The content and meaning of those emails is in fact an entirely different beast from the validity of AGW, which means that experience that's relevant to one is not to the other.

As for the idea that I'm a proxy for AGW, let me say this clearly: I'm not. I'm merely trying to show how arguments on the other side are flawed. If you want to understand this stuff, go to your nearest research university, enroll in some of the relevant science classes, and argue this stuff out with your professors in office hours. They have the knowledge to answer your questions, and they're getting paid to do so.

It's so nice of you, Chris, to try to rule me out of discussing the topic until I have educated myself to your satisfaction. It is particularly charming that you assume my level of education in the first place, and that you assume that my attempts to search out data are the acts of a naif, or that I perhaps do not understand the idea of proxy data. Thanks for the illumination.

Less sarcastically, you seem to have utterly missed the point of my references to your argument technique, so allow me to elucidate. You see, by labeling your argument, showing how you were constructing your points by invoking invalid techniques, I was showing that you are not arguing in good faith. It's not that you are trying to reach some conception of truth; you're simply trying to win the argument. That is to say, you are making a political argument while claiming it's scientifically valid, based on ... wait for it ... the very evidence whose validity has just been severely challenged.

Not smart, but then I've come to believe that your alleged doctorate has given you language, but not understanding. Moreover, I'm a little disgusted with myself for allowing you to drag me down here, rather than actually looking into the facts of the matter in more detail. It's just that I hate to abandon the field, such that your mere ability to persist until I quit is somehow evidence that you've won the argument, and in doing so put the actual scientific evidence out of question.

Well, it's not, and I'm not going anywhere, so please, by all means, keep making invalid arguments, and I'll keep shooting them down.

Mark-

The NRC's conclusions drew on the fact that Mann's results mirrored other evidence, which goes back to the house of cards mentality.

A) The NRC's conclusions drew on that fact, but that was far from all they covered. And I've gone into the flaws of the Wegman report elsewhere.

B) Mann's results mirrored other independent evidence. You seem to have this odd idea that the NRC supports AGW only because a bunch of otherwise mediocre data points in the same direction, but the individual temperature reconstructions - the tree ring data, the ice core data, etc. - isn't actually strong enough to stand on its own. In fact, actually reading through the report reveals that the other reconstructions are quite strong on their own, that they aren't believed to be true just because of what Mann says, and that the fact that they do reinforce each other makes the case for AGW stronger, not weaker, as you seem to be implying.

C) The methods in question are not "right/wrong," but rather "good/better/best." This happens a lot, especially when scientiists are essentially inventing the methodology to answer a question during the course of their research. Mann has since published papers which completely remove any of the work he was (incorrectly) criticized for, and it still points in the same direction. Hanging on to M&M's attacks for dear life as a magic bullet which will stop AGW in its tracks just isn't working, Mark.

As for your IRS analogy, it doesn't hold water when comparing data across multiple independent cliamte research groups. It's like saying that because I have tax issues, everybody else I work with also has tax issues. Doesn't fit.

FYI, tree ring data can be found at NOAA. I haven't yet determined if it can be downloaded en masse instead of just through the search engine, or whether it is raw or massaged data. It looks raw, and it looks to be well enough documented to be usable. I was reading one of the links Chris provided (this one), and it has a fair bit of useful information. For now, I'm not trusting the numbers until I've had a chance to look into them in depth, but the text is quite useful.

The paper seems to do a reasonable job of laying out how to interpret the tree ring/composition data, and where and how it alters based on a number of factors. I did not see the particular formulae necessary to calculate corrections to get estimated temperatures, but assuming that the formulae are available somewhere, that the data noted above is raw, and that the backing research cited in the paper is valid, then I should be able to reproduce the hockey stick's tree ring portion. I note that the paper spends some time on why the records are potentially unreliable for dates after, IIRC, 1980, in at least some of the sites it discusses, but seems to assume that such factors were not present in past records. However, this may come into play in error bars in the formulae, once I find them, based on the hockey stick graph showing larger error regions (HUGE error regions) for older data.

FWIW, I will continue to look.

Benzopf-

It takes an extreme act of mental double-jointedness to justify the active stifling of debate by these scientists by cloaking it in comfortable terms of process. This is beyond the pale and postiviley Orwellian; threats and intimidatation are what they are, and there's nothing usual or scientific about it. Your defense you perform these remarkable feats of intellectual dishonesty, apparently to your own satisfaction and with a straight face, tells me all I need to know.

Honest debate presupposes at least a passing adherence to available facts. Further debate with someone this blind to facts- papering over, denying or ignoring them- is both unwarranted and counterproductive.

Chris, with your post above, you put yourself outside legitimate debate.

So, you're commenting on one of my posts to tell me... you won't be commenting on my posts?

And because you don't like my arguments, you're labelling them "Orwellian" and trying to shut me out of the debate?

Sounds rather... Orwellian to me, Benzopf.

That said, let's put this in context:

- Would you have a problem if this was the field of medicine, and the scientists in question were protesting an editor who choose to publish articles about vaccines causing autism, or HIV not being responsible for AIDS?

- Would you have a problem if this was the field of geology, and the scientists in question were protesting an editor who choose to publish articles about the "vapor canopy" and other aspects of Young Earth Creationism?

- Would you have a problem if this was the field of biology, and the scientists in question were protesting an editor who choose to publish articles about Intelligent Design?

You may not agree that climatologists protesting the inclusion of these papers is the same thing, because you agree with articles skeptical of AGW science, but they clearly do think having an editor who'll promote this kind of junk science is an actionable offense. And unless you're prepared to say that they shouldn't have the right to follow their conscience and make their arguments to the publishers, I don't think you have much of an argument as to why their actions are somehow deeply unethical or immoral unless you a priori assume that they are acting in bad faith and shutting out debate that they don't really believe to be junk science.

In which case, if you're making that assumption, there's not much chance of talking to you about this, either.

toc3-

I would like to speak to you about something I am working on. If I could send my Email to you through AL or vice-versa, I would appreciate it.

I tend to avoid moving into email discussions from comment threads as a rule, and I frankly spend far more time than I should on these debates already. That said, if you'll have AL add your email to your comment handle, I can email you. If not, just comment to that effect and we can arrange something through AL.

I certainly wouldn't want to see the ID proponents at least pushed out. For one thing, science should be challenged, and while ID is crap, they have come up with some interesting points that required a while to formulate good responses to. That makes the theory (evolution, in this case) stronger. And sometimes the crackpots succeed, and everyone starts thinking black holes just might be real. But for another thing, crap like ID and Young Earth Creationism quickly fall flat on their face as science, and that exercise is also useful.

But far more important than all of that is the clever wallpapering you attempted. You see, people working on evolution don't hide their data and models, claim to have lost their original data (which might be worse, because it looks like the AGW people really did lose or "lose" their original and irreplaceable data), attack their critics for wanting the data and methods to study, and tell everyone to just trust them. Moreover, I have yet to see a biologist suggest that evolution means that we should impoverish the entire developed world in order to prevent the possible future in which giant termites devour all humanity. Which means, of course, that the situations aren't comparable.

But nice try. Please try again soon.

Well, it's not, and I'm not going anywhere, so please, by all means, keep making invalid arguments, and I'll keep shooting them down.

Jeff, my arguments stand on their own. That said, with the holidays coming up, I should probably retire from the field, since I'm not sure how much more there is to say - I'll repeat myself four or five times, but six is pushing it.

I'm gratified to see you making use of my links (and I'm not exactly clear how I could "rule [you] out of discussing the topic" when I pointed you towards virtually every link you might want on the subject of AGW), and I look forward to AL putting a crow up on the front page.

That's all, folks.

Whoops, looks one slipped through.

Jeff, I'll ask you to take a look at the history of arguments against evolutionary science in the US; how many biologists have consistently been accused of faking fossil data or not understanding basic science. Far from "quickly falling flat on their face," junk science like creationism has lasted for decades, and will likely be around for decades more. Perhaps science is made stronger by having to debate this stuff, but said debate doesn't happen in the context of scientific journals, nor do any of the evolutionary biologists I'm aware of want it to.

You might also take a look at the branches of conservatism that are "still quite convinced evolution is a lie": http://arstechnica.com/old/content/2008/06/conservapedias-evolutionary-foibles.ars , and how ticked off and uncooperative actual scientists get when they're attacked by people who don't understand the science in question. You might even notice how evolutionary biologists such as Richard Lenski has been accused of the exact same claims regarding withholding and/or falsifying data that you're currently leveraging against climate scientists. (Although in both cases, the vast majority of the data actually is available - in climate science, as evidenced by the links I've provided throughout this thread. Whatever errors have been dredged up by the CRU hack don't invalidate AGW any more than Piltdown Man invalidated human evolution.)

As for "impoverishing the entire developed world," not only is that claim completely ludicrous on its face (we pay vastly more in interest under the debt run up under Republican presidents over the past 30 years than we'll pay in taxes under any suggested cap-and-trade legislation - somehow the economy is still functional), but it again demonstrates ignorance of the history of evolutionary science vs. creationism. Creationism proponents have been arguing for decades, if not centuries, that broad acceptance of evolution will destroy belief in God, damn millions to hell, and ultimately lead to the downfall of Western civilization.

In comparison, your complaints about AGW leading to slightly higher taxes is weak tea.

And now I'm well and truly out.

Chris -

I may be distracted this weekend by a command performance (reviewing someone's white paper), so slower than I'd planned on reviewing the 'availability' of core data.

But here's a snip from an email by Judith Curry, who appears to be the head of the environmental sciences department at Georgia Tech (just for credibility's sake):

The HADCRU surface climate dataset and the paleoclimate dataset that has gone into the various “hockeystick” analyses stand out as lacking such transparency. Much of the paleoclimate data and metadata has become available only because of continued public pressure from Steve McIntyre. Datasets that were processed and developed decades ago and that are now regarded as essential elements of the climate data record often contain elements whose raw data or metadata were not preserved (this appears to be the case with HADCRUT). The HADCRU surface climate dataset needs public documentation that details the time period and location of individual station measurements used in the data set, statistical adjustments to the data, how the data were analyzed to produce the climatology, and what measurements were omitted and why.
Digesting...

I may be distracted this weekend by a command performance (reviewing someone's white paper), so slower than I'd planned on reviewing the 'availability' of core data.

This is not an argument, AL, just a comment: pathetic. You don't need to put "availability" in quotes, and it shouldn't take more than five minutes browsing to confirm that there's far more data out there than you wrongly assumed. Nor do I intend to debate you when you inevitably come up with some nonsense reasons why the data and models don't count as "core".

Put up the damn crow, or admit you won't do it. End of story.

Chris, there's a lot of data; no one's doubting that. The problem is that there doesn't seem to be a lot of raw data: everything I have been finding so far (haven't looked into the tree ring data in depth yet) seems to be massaged data. Well, if you want to validate the massaging data, you have to start with the unprocessed, ugly, messy, noisy stuff, not with the massaged data. And since it is the validity of the massaged data that has been called into question, the only way to tell is to look at the raw data and how it was massaged.

It's actually, by the way, not the much, much (not "slightly") higher taxes I'm worried about, by the way. It's the much, much higher level of government intrusiveness into everything I do that is justified by AGW if it is accepted as the basis of policy. Just as I object to the income tax less for the tax than for the power it gives government to control much of how I work, how I am paid, whether and how I can open bank accounts and the like, I object to measures like cap and trade less for the cost (though I do object to that) than for the regulations and losses of freedom it inevitably must entail.

"B) Mann's results mirrored other independent evidence."

How is that exculpatory when he is being accused of goosing data to match other evidence?

"You seem to have this odd idea that the NRC supports AGW only because a bunch of otherwise mediocre data points in the same direction, but the individual temperature reconstructions - the tree ring data, the ice core data, etc. - isn't actually strong enough to stand on its own."

No- you are the one expanding the scope. I am the one arguing that it is improper to do so- each study needs to be internally consistent. Its one thing to compare your results at the end of the day to other studies, quite another to blunder through your methodology and then claim that because the result it consistent with other studies the methodology is immaterial. That way lies madness.

"In fact, actually reading through the report reveals that the other reconstructions are quite strong on their own, that they aren't believed to be true just because of what Mann says, and that the fact that they do reinforce each other makes the case for AGW stronger, not weaker, as you seem to be implying."

A flawed study doesn't add its weight to the validity of similar studies by nature of its similar results, particularly when cherry picking is suspected.

"Hanging on to M&M's attacks for dear life as a magic bullet which will stop AGW in its tracks just isn't working, Mark."

And building strawmen isn't either. YOU appear to be the one attempting to make that conclusion. All I am arguing is that that study was strangely flawed and there was a coverup to try to prevent that from leaking out. All of this documentation supports that and opens questions about other studies. MY only argument is that this clique has lost their right to the benefit of the doubt, which no scientist should have in the first place. Show your work, release the data and the methodology so others can reproduce it. I find it LUNATIC that you are arguing against this.

"As for your IRS analogy, it doesn't hold water when comparing data across multiple independent cliamte research groups. It's like saying that because I have tax issues, everybody else I work with also has tax issues. Doesn't fit."

You're the only one talking about multiple groups (just how independent they are readers of the emails can judge). All you keep saying is that because there are reams of other data out there, nobody should worry about the raw data that IS missing.

If you claim there is no missing data, where are these HADCRU datasets? And please don't point me to OTHER datasets that aren't missing, that a pathetic slight of hand.

Mark,

In fairness to Chris, it doesn't look like lunacy. It looks like religion.

Something I noticed a long time ago is that every religion has what I call "marker beliefs:" things that are so silly that no one can possibly believe them (like one equals three) literally, but professing them marks you as one of "us" and not doing so as one of "them." For the AGW types, apparently arguing that science requires hiding and falsifying data, and correlation is causation, are required marker beliefs.

Chris - to quote someone I kind of admire, I'm no one's trained monkey. I am obviously interested in this and intend to do some digging; I'll do it on a timeline that's convenient to me. Feel free to remind me every few days.

OTOH, when the head of the department of Earth Sciences at Georgia Tech accuses the AGW community of "lack of transparency" that's kinda suggestive, isn't it?

Marc

the head of the department of Earth Sciences at Georgia Tech accuses the AGW community of "lack of transparency"

That is a very misleading statement. Curry, in fact, is writing about a lack of transparency not, as you say, in the AGW community but rather in climate data--and not climate data in general, but in two particular datasets, one of which she states has already been made more transparent. Furthermore, as a member of the AGW community, she is urging this transparency because it will offer a more effective means of convincing the public of the reality of AGW.

For example, in the same email, she comments that "Gavin Schmidt’s current efforts at realclimate are a good step in the right direction of increasing transparency."

Here's another snippet from her email, that I believe is more representative of her overall views of the AGW community than your comments:

The motivation of scientists in the pro AGW tribes appears to be less about politics and more about professional ego and scientific integrity as their research was under assault for nonscientific reasons (I’m sure there are individual exceptions, but this is my overall perception). I became adopted into a “tribe” during Autumn 2005 after publication of the Webster et al. hurricane and global warming paper. I and my colleagues were totally bewildered and overwhelmed by the assault we found ourselves under, and associating with a tribe where others were more experienced and savvy about how to deal with this was a relief and very helpful at the time

In fairness to you, I should point out that you provided the link to the full email. It's just that your use of it and characterization of her "accusation" is misleading.

Chris-

I can't prevent you from debating, I'm merely pointing out that you are attempting to defend the indefensible and that one can draw certain inferences from that. I'm not even talking about the science here but the actions of these scientists to:

actively stonewall attempts to examine data and models for the purposes of replicating results

destroy or "lose" data when faced with the prospect above

blackball other scientists from publishing, creating the illusion that the "science is settled"

engineer through intimidatioin the removal of people who either hold or are in a position to publish alternative views

These are all facts in evidence, admissions made by those involved. When you deny them in a spectacular act of mental evasion (talk about deniers ! ), you undercut any claim you may have had to objectivity. You show yourself to be a true believer and you show me that there is no behavior so low, so unscrupulous on its face, that you're unwilling to defend it.

(A little off topic here but, just as a practical matter, you might want to consider not co-opting one side of the debate lock stock and barrel, right or wrong, valid or specious. It's just not smart)

We aren't talking about psuedoscience or even other fields of science. Your analogies aren't valid if you can't show that opposing views are not good science. A clique of like-minded researchers, each peer-reviewing one another's work, doesn't qualify as an idictment on the work they conspired to block, BTW.

The actions of the CRU "scientists" suggest only one thing: fear of debate and fear of opposing views. Which as others have pointed out suggests fear of the validity of their own science.

We aren't talking about psuedoscience or even other fields of science. Your analogies aren't valid if you can't show that opposing views are not good science

But the people in question believe, in good faith, that it is junk science, and they believe that they have adequately demonstrated that it is junk science. In that case, Chris's analogies comparing this to the battle between biologists and creationists are valid. The problem is that you don't believe it is junk science. But these people need to act according the their judgments, not yours.

The actions of the CRU "scientists" suggest only one thing: fear of debate and fear of opposing views

No, not the only thing. Their actions can also suggest that they no longer want to participate in what they feel is a politically-motivated, non-scientific debate which they believe, in good faith, has already been settled. Now, if they held your views about the science, then you could maybe claim they feared debate. But since they don't, you can't.

"That is a very misleading statement. Curry, in fact, is writing about a lack of transparency not, as you say, in the AGW community but rather in climate data--and not climate data in general, but in two particular datasets, one of which she states has already been made more transparent"

I agree with you to a degree, but I think you are underselling what she is saying by indicating she only means two specific instances as opposed to a more widespread reluctance to let both data and metadata reach the light of day.
From her email:
_"Climate data needs to be publicly available and well documented. This includes metadata that explains how the data were treated and manipulated, what assumptions were made in assembling the data sets, and what data was omitted and why. This would seem to be an obvious and simple requirement, but the need for such transparency has only been voiced recently as the policy relevance of climate data has increased."_
...
_The need for public credibility and transparency has dramatically increased in recent years as the policy relevance of climate research has increased. The climate research enterprise has not yet adapted to this need, and our institutions need to strategize to respond to this need._

Without doubt her concern is the public integrity climate scientists are viewed with. I think we all are.

And again I have to reiterate that its not the existence of AGW that is the most relevant issue at stake here- it is the claim that catastrophic global warming is imminent. Simply proving that the world has warmed doesn't prove that larger claim- more extraordinary evidence is required and that just happens to be what Mann et al have supposedly produced. There ISN'T an overwhelming amount of evidence for cataclysmic AGW, so examining such studies is critical.

Mark B.,

Without doubt her concern is the public integrity climate scientists are viewed with

Yes. True. But I would point out that she explicitly ties the need for more transparency in climate science to it's relevancy to public policy. She's arguing for this as a tactic to promote the claims of the AGW community. She doesn't seem to doubt the validity of the claims or the accuracy of the data that supports the claim.

I am not qualified to evaluate the evidence of global warming any more than I am qualified to evaluate the effectiveness of a flu shots or air-safety regulations. At some point, I am willing to trust the experts, understanding and accepting that they might be wrong, but believing they act in good faith. I don't believe anyone actually wants cataclysmic AGW.

"Yes. True. But I would point out that she explicitly ties the need for more transparency in climate science to it's relevancy to public policy. She's arguing for this as a tactic to promote the claims of the AGW community."

I agree, but that's a knife that cuts both ways! There is a tremendous fallacy out there that every (or even most) climate scientist agrees with the 'end of world' scenario. I think everybody wants to see where the evidence leads, and everybody wants it for public policy. That could mean that you heartily agree that there is AGW, but that it won't be a major issue for another hundred years. That has huge policy implications. The 'claims' of the AGW community are that the earth is warming. How severely and what the means for mankind and what we should do to stop it given limited resources is a hotly debated topic- or should be.

Mark-

That belief in good faith (that the skeptical science is junk) appears to be founded on dubious science- now that we know the CRU group have shown themselves to be willing to go to extraordinary measures to fudge data and stifle debate.

To continue to defend the work of men who have demonstrably chosen to abandon scientific method, well, that's not good faith, that's blind faith. Personally, I think such people have no business impugning the work of those that do follow the rules while defending those who pervert them, but that's just my opinion. Certainly, one forfeits any pretensions to intellectual honesty.

Using ID and Creationism as an analogy to the work of MM vis-a-vis peer review is worse than nonsense. Those pseudo-sciences cannot be tested; MM's work, in a sense, was not allowed to be tested. Peer review is shown to be flawed if it can so easily be perverted, which frankly throws all of the literature into doubt. But that's another discussion.

The CRU work may not be disproven by all this, but it can no longer be considered reliable (if it ever was)- in fact, any conclusions based on the current work should be considered invalid, at least until the raw data and the models are reexamined. Minus the cabalistic shenanigans.

But, by all means, keep defending this garbage.

And to follow up on that- this exact clique involved in these emails are by far the most influential leaders of the 'end of the world' scenarios. They are the guys that wrote the UN IPCC studies.

Meanwhile there are many MANY 'normal' climate scientists, geologists, etc that certainly agree with the evidence that the earth is warming, but are not convinced it is an imminent danger that these guys are claiming.

Above comment Benzopf #159 in reply to Mark #155

AL,

when the head of the department of Earth Sciences at Georgia Tech accuses the AGW community of "lack of transparency" that's kinda suggestive, isn't it?

Is it as suggestive, when she says this:

"You cannot blame any single storm or even a single season on global warming. ... Gore's statement in the movie is that we can expect more storms like Katrina in a greenhouse-warmed world. I would agree with this," said Judith Curry. She is chairwoman of Georgia Tech's School of Earth and Atmospheric Sciences, and is co-author, with Mr. Webster, Mr. Holland and H.R. Chang, of a paper titled "Changes in Tropical Cyclones," in the Sept. 16 issue of Science, a weekly publication of the American Association for the Advancement of Science

Although that is from the Washington Times (the ellipsis is theirs), so I can't vouch for its accuracy. Here's a another quote.

"The best available data shows that the number of Category 4 and 5 hurricanes globally has almost doubled since 1970," said Ms. Curry at Georgia Tech, in an e-mail response to questions. "In the North Atlantic, there has been a comparable increase in intensity, and also a 50 percent increase in the total number of North Atlantic hurricanes. This increase in hurricane activity has been linked to a 1-degree Fahrenheit increase in global tropical sea-surface temperature since 1970. This global temperature increase since 1970 is attributed to global warming."

Still suggestive? Or are we less fond of her now?

Of course, she's not saying global warming is man made. It could be those sun spots. It could be that increased greenhouse gases in the earth's atmosphere do not have any impact on the earth's temperature.

"Of course, she's not saying global warming is man made. It could be those sun spots. It could be that increased greenhouse gases in the earth's atmosphere do not have any impact on the earth's temperature."

Are you suggesting it couldn't be? Hasn't the earth been this warm and warmer in the past?

For the record, CO2 makes up .0387% of the earth's atmosphere. So do I think it's reasonable that mucking with that trace amount of gas isn't sending something as enormous and complex as the earths atmosphere into thermal meltdown? Yeh, i think thats plausible.

Mark B.,

I'm reluctant to get too deeply into this as I am as far from being an expert in this subject as I am from being a major league pitcher. However, it is my understanding that this statement:

They are the guys that wrote the UN IPCC studies is not true. They are among dozens of authors who wrote these reports.

Also, I believe that while this : there are many MANY 'normal' climate scientists, geologists, etc that certainly agree with the evidence that the earth is warming, but are not convinced it is an imminent danger that these guys are claiming

is true enough, at the same time, I believe that most of these other 'normal" climate scientists--or at the very least, the scientific bodies that represent them -- have endorsed the IPCC reports, knowing full well who wrote them and knowing all about the surrounding hockeystick controversies. Again, I just have to trust the experts. It appears to me that there is a consensus among them.

I do not at all disagree with you that what the consequences of global warming might ultimately be, over what time period, and what we ought to do or not do about it, is a legitimate subject for debate. I do not agree that we should wait for conclusive proof before acting. I would have urged people to quit smoking, e.g., while there was a mere correlation between smoking and lung cancer but not conclusive proof of causation.

Mark B.,

Hasn't the earth been this warm and warmer in the past?

I'm sure it was. I don't think anyone is concerned about the earth's health, so much as our own. Just as an example, there's a lot of wheat planted around the world, with a lot of agricultural infrastructure specifically designed to support the production, in particular latitudes because, in part, of those latitudes' average temperatures. If there is significant enough warming, all of that may have to be replaced by, say cotton. That may not be catastrophic in your eyes, but it will be disruptive and expensive.

For the record, CO2 makes up .0387% of the earth's atmosphere. So do I think it's reasonable that mucking with that trace amount of gas isn't sending something as enormous and complex as the earths atmosphere into thermal meltdown?

I don't know. I'm no expert. But it takes less than 400 parts per 1,000,000 of arsenic to kill you. And, it doesn't take a thermal meltdown to be a concern. A few degrees difference one way or another can have enormous economic consequences.

Look, I'm not afraid I'm going to die. But I do live on an island off the coast of the US and I'd rather not have sea levels rise even a few feet because of the damage that would do to the building where I work.

It seems now that you have gone from arguing that the issue is not how catastrophic AGW might or might not be, but whether or not there is such a thing.

As far as we can tell, Harry gave up (link).

The file peters out, no conclusions. I hope they find this poor guy, and he didn’t hang himself in his rooms or something, because this file is a summary of three years of trying to get this data working. Unsuccessfully.

I think there’s a good reason the CRU didn’t want to give their data to people trying to replicate their work.

It’s in such a mess that they can’t replicate their own results.

This is not, sadly, all that unusual. Simply put, scientists aren’t software engineers. They don’t keep their code in nice packages and they tend to use whatever language they’re comfortable with. Even if they were taught to keep good research notes in the past, it’s not unusual for things to get sloppy later. But put this in the context of what else we know from the CRU data dump:

1. They didn’t want to release their data or code, and they particularly weren’t interested in releasing any intermediate steps that would help someone else

2. They clearly have some history of massaging the data — hell, practically water-boarding the data — to get it to fit their other results. Results they can no longer even replicate on their own systems.

3. They had successfully managed to restrict peer review to what we might call the “RealClimate clique” — the small group of true believers they knew could be trusted to say the right things.

As a result, it looks like they found themselves trapped. They had the big research organizations, the big grants — and when they found themselves challenged, they discovered they’d built their conclusions on fine beach sand.

When your work is chicanery piled on fudging piled on manipulation so high that you can't redo it yourself, then systematically stifling potentially critical review doesn't even carry the weak shield of a "good faith" excuse.

The critical information that was never release and couldn't be was that there was no mysterious Part X that made everything work. The project was a fiasco.

Start again. Get new scientists untainted by association with the catastrophic Anthropogenic Global Warming science clique, gather new data, and build credible science.

So, for fun, I decided to go through the emails and count word frequencies, just to see what came up. Here are the top 30 word frequencies (ignoring 1 and 2 character words and common, meaningless words like 'the'):

4142 data
3888 uea
3378 mann
3343 climate
2728 jones
2257 phil
2163 briffa
2097 fax
2045 university
2011 keith
1922 research
1913 temperature
1862 time
1844 should
1797 think
1678 subject
1657 series
1655 email
1579 date
1574 other
1506 science
1489 wrote
1420 1603
1411 model
1407 cru
1389 could
1333 their
1320 may
1308 see
1300 work

The "1603" is from the telephone numbers, by the way.

Chris - to quote someone I kind of admire, I'm no one's trained monkey. I am obviously interested in this and intend to do some digging; I'll do it on a timeline that's convenient to me. Feel free to remind me every few days.

Whatever, AL. It takes five minutes to verify whether the links I provided are the evidence you asked for.

You say you're "interested in this," but the very fact that you couldn't be bothered to spend five minutes to find the data and models prior to your post shows that to be a lie.

You say you'll do it on a timeline that's convenient for you, but you have a long history of not following through with stuff - this site is littered with posts "you're working on" that never materialize.

Given that history, the fact that that you haven't followed through on this four days later is excellent indication that you never will. And that's ok - it's excellent proof for my central thesis about the nature of your arguments.

Post the crow or not, AL. I've seen all I need to see here.

I think what Chris means is, "Dance, monkey, dance." It's a shame he hasn't learned to be civil when a guest at another person's place.

Chris, you've got to be kidding.

What I asked was whether the underlying data and models for the "foundational" papers (a concept I hope is clear to all) in pro-AGW climate science were publicly and readily available.

That implies two things...

1) looking through the literature to establish what the foundational papers might be;

2) reviewing the availability of the core data and code - and to do that effectively, looking at the commentary on the papers to see if they call into question the availability of data and code.

Sorry, but clicking through six links that you proffer doesn't cut it.

Particularly when people who are grown-ups in the area of climate science - and even pro-AGW scientists - complain about the lack of transparency in the basic science.

The news today that the NZ data may have been fudged - and has been unavailable until recently - is another call on this.

So do your little chest-beating dance or not; it signifies nothing.

I still intend to do this in the near future (maybe this weekend, depends on emergent priorities); if you want to put up a taxonomy of papers that suggests which are foundational, you're welcome to contribute. Or you're welcome to caper...

Marc

David Blue (#166)

Start again. Get new scientists untainted by association with the catastrophic Anthropogenic Global Warming science clique, gather new data, and build credible science.

I'd vote for it, but please check there might be some problems with that:

A scientist beyond a brain has a stomach. The last, unlike the first, must be filled an average of three times a day. After so much cash poured into Global Warmology there might be a few non-believers still exercizing as climate scientists. Moreover, studies (and movies) about catastrophic events are easier funded.

The problem with data is that there not exist. If it had existed, no deception could have taken place. That is the base of deceptions and the contrary is illogical.

I think we should study whether science, alone, science, can be built in this corner of human worries. IMHO the core problem that has given birth to Global Warmology have been computers. Climate scientist have launched themselves to program global scale models simply because computers have grown powerful enough to calculate them. But a computer model, beyond the equations, beyond the maths, needs data. Data not only to feed or validate it, but to tune it.

And we don't have data. We have just some information from the last 35 cycles of a system that has been working for billions. Before the 1970's and the explotation of space for these porpouses, there was no way to assess the weather globally (and hence the "increase" in hurricanes thereafter).

That is no way to feed, validate or tune a computer model, not even one that predicts small changes. If great changes are to be predicted, then the issue worsens: the condition to validate the model would be whether the results from the model, fed with the same conditions, fit a past event of its kind. For instance, a car computer model might fit pretty well its behaviour driving straight and fail when trying to study a sharp turn because in a sharp turn conditions that kept constant in the first case no longer are in the second. But the fact is that we don't have global data for sharp weather changes, which inevitably leaves Global Warmology as one of the most succesful religions of the late 20th-early 21st century.

What I asked was whether the underlying data and models for the "foundational" papers (a concept I hope is clear to all) in pro-AGW climate science were publicly and readily available.

That implies two things...

1) looking through the literature to establish what the foundational papers might be;

2) reviewing the availability of the core data and code - and to do that effectively, looking at the commentary on the papers to see if they call into question the availability of data and code.

AL, I foresaw way back comment 69 that this would turn into an argument about what the "core" data sets represented. I was hoping to avoid this, but let me put this as simply as possible:

1) You didn't ask for the "foundational" papers, you asked "Have the core AGW datasets and code been released for general review, and if so where?" In this context, the core data sets and models are those that are central to the IPCC 4th Assessment Report, which is the central driver of political action on AGW. I've already linked to the data from the 4th AR; the list of 23 models used in the 4th AR can be found on pages 597-599 here. I've already linked to at least four of them above; others, like the ECHAM5 model, are trivially easy to find

(While I'm at it, note that "foundational" as it is used in social sciences is not nearly as relevant in the physical and engineering sciences - in the former, one or more key papers are released which essentially kick off an entire discipline which future researchers then spend decades filling out. In the physical sciences. papers don't so much kick off disciplines so much as identify new questions to ask, and the most important research isn't the earliest papers asking those questions so much as the latest papers answering them. Note, for example, that the NRC paper focuses far less time on the origins of disciplines such as paleoclimatology, and far more on the latest research from those fields.)

2) You said several times above that the models had "never, to my knowledge been released to be peer-reviewed." It is trivial to verify that "never been released" is a lie - clicking through the links I've provided really is all that's needed to verify that those models are available for review. When you start searching out commentary for reasons to doubt the validity of those models, you're not talking about verifying availability, you're talking about actually reviewing the models. That' was not the point under contention w/r/t the crow banner, and it is typical that you'd try to obfuscate the issue with non-sequiturs such as your status as a trained monkey. (Or misrepresentations about what Judith Curry said, which mark already debunked in #153.)

You're trying to portray yourself as someone who's committed to investigating the truth of AGW in his own time, AL, but, as I've said before, your own record of false statements regarding models and data, your own profound disinclination to search out anything but reports that back up your preexisting beliefs, and your lackluster record of following through on projects makes it unlikely that you'll ever get there.

But it is, as ever, amusing to see you try to convince yourself and others that it's otherwise.

Sure there are foundational papers in physics and engineering, and pretty relevant, by the way. For instance, we have this one, which makes your mobile phone work. The Cooley-Tukey algorithm allowed the implementation in digital devices of the Fourier transform, both answering questions and opening a new field for research.

Now the question is, where are the foundational papers of Global Warmology? Where are they? In which paper a scientist said: look, you see the last glaciation, I input the global data during the previous years of that event and my model is able to predict what happened thereafter. We could expand its use...

If NASA spends 300 million of your bucks launching a prototype plenty of sensors in order to validate its models, models being improved during more than half a century and still unable to predict the behaviour of a rocket in such configuration, why do Global Warmologist skip this key step?

A friend of mine would say that because they don't have to fly on their models.

ure there are foundational papers in physics and engineering, and pretty relevant, by the way. For instance, we have this one, which makes your mobile phone work. The Cooley-Tukey algorithm allowed the implementation in digital devices of the Fourier transform, both answering questions and opening a new field for research.

Now the question is, where are the foundational papers of Global Warmology? Where are they? In which paper a scientist said: look, you see the last glaciation, I input the global data during the previous years of that event and my model is able to predict what happened thereafter. We could expand its use...

If NASA spends 300 million of your bucks launching a prototype plenty of sensors in order to validate its models, models being improved during more than half a century and still unable to predict the behaviour of a rocket in such configuration, why do Global Warmologist skip this key step?

A friend of mine would say that because they don't have to fly on their models.

Chris, you are such a tool, you amaze even me.

Presented with someone who would - with half an effort - be open to serious, validatable, claims that - contrary to the commentary by people ranging from Judith Curry to Eric Reynolds - the core models and data are available - you present links that (on first glance appear to be) to massaged data that it itself the 'corrected' output of the kind of models that we're criticizing here.

I'll make that review in a future post, once I finish pulling out cites from the IPCC history.

And as to 'follow-through'; donno, we raised and gave a couple grand away - not what I'd hoped for, but not nothing.

But as long as we're dick-measuring, in the last few years, I can say that I've convinced a politician that voting technology was an issue - she later ran on that platform and started the fix here in California, which spread nationally; helped grow a charity that sends aid to Iraq and Afghanistan from a car-trunk operation to a several-million-dollar a year operation that has helped tens of thousands; helped pass badly-needed school bonds in my own city; convinced at least five people to run for office and supported three to victory, one to a loss, and one to be determined soon; helped run an arts company that gets international recognition; maintained a pretty substantial body of professional activity (and made a living doing it); pretty much finished raising two great kids and am about 2/3 through with one; been happily married; had a great set of friends. I got to ride Laguna Seca, too.

And in between all that, I've managed to bitchslap about a dozen moronic trolls like you until they've gotten bored and left.

Now you have a choice; you can stop being a troll, participate in the discussion and maybe, just maybe win some converts. Or you can jump up and down squealing, enjoy being a dick while the grownups all giggle behind their hands and whisper "what a dick..." and try to do real work.

What's it going to be, kiddo?

Marc

Ok, let's try this:

...you present links that (on first glance appear to be) to massaged data that it itself the 'corrected' output of the kind of models that we're criticizing here.

Really? Then what the hell is this tar file , which is located all of two clicks away (under the big blue SOFTWARE tab no less) from the NASA archive I linked to earlier?

Do you think you're making a particularly good case for yourself as someone who's "open to serious, validatable, claims" when you can't be bothered to do even a cursory exploration of the available evidence? Or do you think if you keep reciting "massaged data, massaged data" long enough, it'll magically turn in to something other than the source code you assured us had never been released for review?

Call me a troll, condescend to me, "bitchslap" me (nice language, btw), whatever you like, AL. This kind of utterly asinine argument - which has been in evidence throughout the thread - should give anybody all the context they need to judge the greater validity of your posts here.

And if you have such a wonderful offline life, AL, then I sincerely wish you the best in it. I just don't think you're doing all that hot here.

Really? I'm having a good time and that's what this place is about for me.

It's simple. There ought to be a traceable chain between "claims" > "research" > "models" > "data"

Now, on a dumb level, given that all kinds of people are making the claim that at some level the transparency is missing - it seems at least a little suggestive.

For me - before I spent tens of thousands of dollars 'remediating my carbon footprint' I'd want to see a relatively solid and visible armature for claims that it mattered (note that I have spent tens of thousands minimizing my dependence on energy, with the intent of shielding myself from energy price shocks and from the strategic consequences of national energy dependence).

So you'd assume that such an armature would logically exist somewhere; and yet no one can point to it.

So my goal - largely driven by this discussion, although as a side note you're a disincentive rather than an incentive - is to sketch one out as a way of 'checking the work' of people who - at best - now seem untrustworthy (even in the words of someone from UEA itself ).

Now it may seem foolish to suggest that mere citizens should be allowed to at minimum validate the existence of data validating claims of advanced science; for that I'll again refer to you Rittel (and, actually, Habermas). You don't like it? Pound sand or participate, it's your choice.

And the East German judge's vote just never gets taken seriously; so when your opinion of me matters - I'll let you know.

Marc

It's simple. There ought to be a traceable chain between "claims" > "research" > "models" > "data"

First, the chain is research > models > data > claims. (Nice scare quotes, btw.) If you don't understand that, that might explain in part why you haven't been finding what you're looking for.

Now, on a dumb level, given that all kinds of people are making the claim that at some level the transparency is missing - it seems at least a little suggestive.

So you'd assume that such an armature would logically exist somewhere; and yet no one can point to it.

Second, the IPCC FAR is exactly that; the problem seems to be that you're incapable of seeing that, as evidenced by the fact that you A) didn't realize the models and data were available, and B) couldn't find the freakin' models on a website that was right in front of you.

Third, as for "all kinds of people," we've already covered how you've misrepresented Judith Curry in the past; moreover, what you mean when you (wrongly) say transparency is missing (i.e. that "models have never been released") is substantively different than what she means when she talks about data not being available (i.e. that not every piece of data in the entire universe is instantly and totally available - she does recognize that the vast majority of publicly financed research already is available).

Seriously, stop and think about this a moment - in her letter she says:

Scientists claim that they would never get any research done if they had to continuously respond to skeptics. The counter to that argument is to make all of your data, metadata, and code openly available. Doing this will minimize the time spent responding to skeptics; try it!

And yet, a great amount of data already is available, it has been presented to those on this thread, and it has changed absolutely nothing - we can't even agree on whether the data actually exists, apparently. So while Dr. Curry is a fine climate researcher, I do think the actual history of debate on this subject (she hasn't done a great job of doing anything but getting polite pats on the head from the Climate Audit crew either) tends to undercut her naivete that skeptics can be reasoned with in good faith.

Now it may seem foolish to suggest that mere citizens should be allowed to at minimum validate the existence of data validating claims of advanced science...

Yeah, that's exactly what I was getting at by posting over a dozen links to the relevant research, models, and data in this thread, AL. Thanks terribly for not relying on straw man attacks, incidentally...

And the East German judge's vote just never gets taken seriously; so when your opinion of me matters - I'll let you know.

AL, I am entirely 100% certain that nothing I ever say will make a dent in your arguments or your self image; I'm about 99% the same is true of most of the regular commenters on this blog. Insofar as my arguments are addressed to anyone else, it's random passers-by who might otherwise be swayed by your nonsense.

That, and taking apart your arguments is, if not challenging, then at least good practice. "I'm having a good time and that's what this place is about for me."

And I see you've apparently stopped trying to defend your earlier remarks about none of the models being available for review; when does the crow go up?

Chris, I'll use one of the few replies you have left here...

Your chain is causal - as in "research" leads to "models" which are supported by "data" which in turn leads to "claims" - mine is logical as in "the entity [claim]" contains "the entity [research]" which in turn contains "the entity [model]" which in turn contains "the entity [data]". Each of them works, and I'm willing to allow you your causal chain without making a claim that you're ignorant. But I'll remain content with my logical one.

Yes, you've posted links to at least eight or nine datasets and models. So - the question is 'are these datasets and models are themselves enough to base a firm belief in AGW', and thus the policies and costs involved? and 'do the little numbers and letters on my screen represent both the raw data as collected (according to some disclosed process) and are they the entirety of what goes into the models?

Now maybe it's because I'm thick, but just clicking on those links and poking around hasn't yet answered those questions for me.

As I offered a million comments ago, if you want to construct and document that chain, I'm happy to post it here as a full blown post.

Otherwise, I'll just so my kind of ignorant thing of reading the documents, tracing back the studies cited, looking up the studies, and seeing what's available about them - both as raw data and models (my core claim) and in terms of critique (because - shockingly - I'm actually interested in the issue.

So once that's done, I'll present what I find, and depending on what it shows - I may or may not post a banner with a crow on it.

There's other data that suggests that I'm right to make the criticism that I did - here's the full set of quotes from Curry:

First...
In my opinion, there are two broader issues raised by these emails that are impeding the public credibility of climate research: lack of transparency in climate data, and “tribalism” in some segments of the climate research community that is impeding peer review and the assessment process. 1. Transparency. Climate data needs to be publicly available and well documented. This includes metadata that explains how the data were treated and manipulated, what assumptions were made in assembling the data sets, and what data was omitted and why. This would seem to be an obvious and simple requirement, but the need for such transparency has only been voiced recently as the policy relevance of climate data has increased. The HADCRU surface climate dataset and the paleoclimate dataset that has gone into the various “hockeystick” analyses stand out as lacking such transparency. Much of the paleoclimate data and metadata has become available only because of continued public pressure from Steve McIntyre. Datasets that were processed and developed decades ago and that are now regarded as essential elements of the climate data record often contain elements whose raw data or metadata were not preserved (this appears to be the case with HADCRUT). The HADCRU surface climate dataset needs public documentation that details the time period and location of individual station measurements used in the data set, statistical adjustments to the data, how the data were analyzed to produce the climatology, and what measurements were omitted and why. If these data and metadata are unavailable, I would argue that the data set needs to be reprocessed (presumably the original raw data is available from the original sources). Climate data sets should be regularly reprocessed as new data becomes available and analysis methods improve. There are a number of aspects of the surface climate record that need to be understood better. For example, the surface temperature bump ca. 1940 needs to be sorted out, and I am personally lacking confidence in how this period is being treated in the HADCRUT analysis. In summary, given the growing policy relevance of climate data, increasingly higher standards must be applied to the transparency and availability of climate data and metadata. These standards should be clarified, applied and enforced by the relevant national funding agencies and professional societies that publish scientific journals.

Second...

If climate science is to uphold core research values and be credible to public, we need to respond to any critique of data or methodology that emerges from analysis by other scientists. Ignoring skeptics coming from outside the field is inappropriate; Einstein did not start his research career at Princeton, but rather at a post office. I’m not implying that climate researchers need to keep defending against the same arguments over and over again. Scientists claim that they would never get any research done if they had to continuously respond to skeptics. The counter to that argument is to make all of your data, metadata, and code openly available. Doing this will minimize the time spent responding to skeptics; try it! If anyone identifies an actual error in your data or methodology, acknowledge it and fix the problem. Doing this would keep molehills from growing into mountains that involve congressional hearings, lawyers, etc.

So forgive me, Chris, but when a fricking Department Chair in environmental science tells the AGW world they have a transparency problem...might it just be that the problem isn't that people like me are a) too stupid to look; or b) dishonest?

I'm interested in this issue; I've been agnostic for some time and it's time for me to take a look and see if there's something to bring my faith in one direction or the other.

But the 'tells' look a lot more like epicycles than real science so far. Again - I'm not nearly done (I almost have a full list of 18 or 20 papers I'd like to dig into) and am working damn hard to keep an open mind. The problem is that - other than a very few people like Prof. Curry - most of the people on your side of the fence just keep sounding like this ...

Marc

Chris, want to comment on this

As far as I know, I am the person who made the original Freedom Of Information Act to CRU that started getting all this stirred up. I was trying to get access to the taxpayer funded raw data that they built the global temperature record out of. I was not representing anybody, or trying to prove a point. I am not funded by Mobil, I’m an amateur scientist with a lifelong interest in the weather. I’m not “directed” by anyone, I’m not a member of a right-wing conspiracy. I’m just a guy trying to move science forwards. The recent release of the hacked emails from CRU has provided me with an amazing insight into the attempt by Steve McIntyre, myself, and others from CA and elsewhere to obtain the raw station data from Phil Jones at the CRU. We wanted the data that was used to make the global temperature record that is used to claim “unprecedented” global warming. I want to give a chronological account of the interactions. I will reference the email numbers so that people can see the entire emails if they wish. While we don’t know if all of these emails are valid, the researchers involved such as Gavin Schmidt and Michael Mann that clearly indicate that they think they are authentic.
So - someone is looking hard for the raw data (what I'm interested in tracking down) - and claiming as of 11/24 that they haven't been able to get it.

Not dispositive, but keeping my interest high.

Marc

AL, you're kind of all over the place, aren't you? First I'm a troll who you'll "bitchslap" until I get bored and go away, then I only have a few reples left (presumably simply based on your fiat) then you're demanding that I respond to some random guy on Watts' site.

Instead, let's focus on this:

Yes, you've posted links to at least eight or nine datasets and models. So - the question is 'are these datasets and models are themselves enough to base a firm belief in AGW', and thus the policies and costs involved? and 'do the little numbers and letters on my screen represent both the raw data as collected (according to some disclosed process) and are they the entirety of what goes into the models?

No, AL, those were not the questions you asked

3) Have the core AGW datasets and code been released for general review, and if so where?

If I'm wrong, and the code and data has been released, I'll be happy to post a picture of a large crow as the banner on this site for a month.

You're changing the terms of the debate now that it's been demonstrated that you've been proven wrong, and you're not fessing up to the fact that you were wrong in the way you said you would.

This is unsurprising to me.

Beyond that, we're going around in circles - I'm skeptical that your proposed evaluation will come to much for reasons I've repeated ad nauseum, but it's a free country, so go nuts. You're seizing on various comments and posts as proof for your a priori belief that climate scientists are hiding something; I've again pointed out why many of your interpretations aren't correct, but given that even getting you to recognize the availability of some data has been like pulling teeth, I doubt we'll be able to have a productive conversation on the subtleties and realistic difficulties of ever achieving 100% transparency in any field of science.

I'll let you have the last word, AL. Good luck embarking on your exploration of climate science.

This is what science should look like. (link) I hope this catches on.

Achieving this is simple and inexpensive. It is not done by more rigorous enforcement of the Freedom of Information Act, although that would help. It comes from branding “openness” into every link of the scientific research value chain. Public or tax-deductible research funding should be contingent upon maximum transparency.
Scientists and affiliated institutions that will not make the research process as transparent as the end result will be asked to return the money or risk denial of future funds. University accreditation should be contingent not just upon faculty research and publication but by demonstrating policies and practices that champion data sharing. Professional societies and journals should make data sharing a condition of membership and publication. Researchers must be pushed to be more open at every step of their process.
The Royal Society not only makes data sharing a precondition of publication, it provides up to 10 megabytes of free space for supplementary data on its website. Unfortunately, too many scientific societies and publishers are less than rigorous or insistent about openness. Strip them of their tax-deductible status. Make opennes a condition of tax advantage. Of course commercial and proprietary issues can influence the manner of data sharing and transparency. But the East Anglia emails represent an individual and institutional imperative to err on the side of minimal disclosure even as researchers sought to maximise the academic and political impact of their work. That is perverse.
Public interest suggests scientists and their sponsoring institutions be made as legally, financially, professionally and ethically as uncomfortable as possible about concealing and withholding relevant research information.

Why what has been going on is not what ought to be going on: (link).

If the “hockey team” had been required to make their primary datasets and modeling code available for unrestricted inspection, the AGW fraud could never have turned into a political monster. If Michael Bellesisles had been required to make all his primary data open for inspection, the fraud that was Arming America would never have won a Bancroft Prize. Sunlight is the best disinfectant, and full disclosure is the final and deadliest enemy of junk science.

...

krygny Says:

November 25th, 2009 at 8:02 am

Wait just a second. Explain this to me like I’m 12. They didn’t even bother to fudge the data? They hard-coded a hockey stick carrier right into the program?!!

ESR says: Yes. Yes, that’s exactly what they did.

--

...of course, they now claim that crucial primary datasets were “accidentally” deleted.

After reading some of the emails about evading FOIA2000 requests…accidentally, my ass.

...

>I am just wondering: Is AGW really junk science, because one team of experts (let’s say “experts”) did have an agenda?
No, it’s junk science because the agenda has driven systematic data suppression and fraud. It’s not the agenda itself that matters, it’s the breacjh in standards of scientific conduct. Those have been repeated and severe.

...

And remember, when the data conflicts with the theory: “The data is surely wrong.”

Chris,

OK, I've been looking for (what? a week?) now, in the time I've had available, and there's something I cannot find. My first question was what the actual measured temperature data we have shows. I know that this will sometimes be inaccurate, and that measurements might have to be adjusted for many reasons (instrument bias, site conditions, elevation and what have you). I also realize that even thermometer readings are just proxies (in that case, it is measuring transferred energy due to molecular motion, so pretty close to the physical concept of temperature). But I was curious what the temperature records actually say, even with all these caveats.

I might be able to use OPeNDAP to get the raw, unprocessed satellite (MSU) records. It has an apparent incompatibility with the Java VM on MacOS X 10.6, which I've reported, so we'll see about that when it gets fixed.

What I cannot find is either the raw, unprocessed ground station temperature records, or the list of which stations are used for HADCRUT. (I know from the literature that they dropped a number of stations that were not useful for various reasons, but I don't know which ones.)

Do you know where I can find those?

Thanks

Jeff-

I do not off the top of my head; if the IPCC model data repository I linked to in #69 isn't useful, try Real Climate's new data sources page, which seems to have pointers to raw data of various types.

Beyond that I'm out of this.

The link below is a breakdown of Willis Eschenbach's FOIA saga with CRU. Its a fascinating read because it displays the notorious context of the emails in question. It also shows the slight of hand going on, and exactly what has been hidden.

Here

Interestingly, Doug Jones was playing the same game Chris is- in this case he was refusing the FOIA requests by pointing to the GHCN data that is indeed available on the web.

But as Eschenbach says, that is useless without knowing the list of actual sites used to prepare the datasets (HadCRUT3 in this case). There is no way to reproduce the work (which was used in the IPCC) without knowing what data was used and why. In his words:
"Pointing at two huge piles of data and saying, in effect, “The data is in there somewhere” does not help at all."

I think that applies to what Chris is saying as well.

Doug Jones = Phil Jones

So the core dispute between Chris and I is whether the "core" data and models for AGW research are public. Chris says "yes" and I say "where". Chris replied with a set of links, but missing from those links is HADCRUT3 and predecessor raw data and computational code to derive HADCRUT3 from the raw data. IPCC used an earlier version of HADCRUT as the basis for its judgment that global temperatures were increasing.

HADCRUT3 output is available here

The emails have some interesting comments about HADCRUT...

But it looks like the University is going to take steps to deal with that; they posted today that

"CRU’s full data will be published in the interests of research transparency when we have the necessary agreements. It is worth reiterating that our conclusions correlate well to those of other scientists based on the separate data sets held by the National Oceanic and Atmospheric Administration (NOAA) and the NASA Goddard Institute for Space Studies (GISS)," concluded Professor Davies.

I'm going to continue my project, and will post a list of papers that I'll be going through tomorrow or Monday...

Marc

HADCRUT3 isn't the data. HADCRUT3 (and earlier versions) are derivatives of raw data. The derivations needs to be shown, since any error or bias there is going to show up when calibrating any model against HADCRUT3.

If poor Harry's travails aren't enough warning that the derivation should be open sourced (and put through code review) I don't know what would be. If he had such trouble, with all the personnel and resources of CRU available to him, it's a travesty to suggest that an outsider could have repeated the work.

Given the trillions of dollars of 'cap and trade' graft that we're talking about sending to third world kleptocrats via the UN, perhaps a very small fraction of that would be better spent buying off any private data owners, to quell that excuse, and then putting up public servers to host it.

I had not see this article when I wrote the comment above. I don't know whether to laugh or cry:

SCIENTISTS at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based.... The UEA’s Climatic Research Unit (CRU) was forced to reveal the loss following requests for the data under Freedom of Information legislation. The data were gathered from weather stations around the world and then adjusted to take account of variables in the way they were collected. The revised figures were kept, but the originals — stored on paper and magnetic tape — were dumped to save space when the CRU moved to a new building.... In a statement on its website, the CRU said: “We do not hold the original raw data but only the value-added (quality controlled and homogenised) data.”

I guess the 'value-added' would be poor old Harry, and Tim the enchanter before him. I think we can now safely disregard HADCRUT3 and prior as credible data sources.

Yes, this is funny.

Wasn't that Bellesisles' defense as well? That and a flood...

Marc

I've taken a brief foray through the links that Chris provided. Some quick thoughts (some are repeats of recent comments above):

1) Much of the data (don't know about all of it) appears to be pre-processed. For example, the CRU datasets all seem to be monthly averages, on a datagrid of their determination. That leaves the rather obvious question of: where are the daily numbers, and how granular is that data? Hourly? Daily high and low? Where were the measurements actually taken?

2) The article titled: Description of the "TRIFFID" Dynamic Global Vegetation Model, by Peter M. Cox, appears to be a study of how vegetation reacts to climatic forcing. Not about the impact of external forcing on climate. Now, there is discussion about the interaction of vegetation coverage on climate, but this is not nearly the clear-cut climate model and supporting data that "proves" AGW. Way to go Chris.

3) The argument that this has all been "peer reviewed" is ringing increasingly hollow as each day goes by. It is readily apparent from the recent whistle-blower email and data leaks that the peer review process is completely corrupt. To try to wave that flag at this point is tantamount to saying "Look! Shiny!" It is hand-waving of the most despicable kind.

For example: CRU is, in fact, in almost complete control of committee placements for the IPCC. This, combined with the fact that a rather tight cabal of highly regarded climatologists (Mann, Jones, Hansen, etc.) exercised undue pressure on various important climate journals, to the point of forcing resignations and threatening boycotts, basically indicates that the objective standards necessary for proper peer review were non-existent. To put it mildly.

There is ample evidence that the review process during AR4 was a joke. The above-mentioned cabal was able to suppress any dissenting opinions, even from apparent insiders.

Chris mentions the thousands of research articles that "confirm" AGW. This is, in fact, not accurate. What a vast majority of these articles confirm is that some sort of warming is occurring (after presupposing AGW is a fact). And even then, there are questions that have to be answered. Many of these papers end up being nothing more than re-hashes of previously questionable data sets and/or methodologies. "Treemometers" have been most definitively trashed, but tree ring data is used repeatedly throughout the literature. The basic flaw that nobody on the AGW-alarmist side is willing to cop to is this: tree rings are not solely influenced by temperature; they are also influenced by carbon dioxide, nitrogen, and moisture. This makes articles by Briffa (and many others), dependent on the assumption that tree ring growth is primarily (or even solely) driven by temperature, fatally flawed.

Too, the statistical methodologies employed repeatedly by Mann (and again many others) are laughably misused. The claim made by many is that Steve McIntyre (of ClimateAudit fame: www.climateaudit.org, or camirror.wordpress.com) does not have an advanced degree, and therefore doesn't "get" climatology. Gang, he spent his life employed as a statistician with various mining companies. He lived and breathed the types of statistical analyses put to use by the "Hockey Team". He and Ross McKitrick pretty thoroughly demolish the Hockey Stick. And the magical mystery tour that AGW-alarmists attempt to take us on in "refuting" M&M is nothing short of mendacious. There is a thread at the Air Vent that chronicles some of the email exchanges. Captured in this group of emails is an exchange between the Hockey Team and M&M "discussing" M&M's take-down of MBH98, one of the seminal Hockey Sticks used by AGW proponents. M&M show pretty clearly both how flawed MBH98 is, and how misleading the counter-arguments were by the Team.

I could go on, but it's getting late. I know there are a lot of links I could provide. I'll see what I can do about getting them together. I didn't start this with the idea of writing an essay. The ultimate point being--Chris is basically full of crap. His basic MO is to throw a bunch of bullshit at us, in the hopes we either won't follow the links, or won't have enough background to understand his deviousness.

Pwnd.

Oh. And just to "ice the cake": there have been revelations over the last couple of days that other "objective" datasets have been compromised. The latest being New Zealand. This, and some climatologists are admitting to buckling under to the pressure exerted by the Team.

Nice.

"Many of these papers end up being nothing more than re-hashes of previously questionable data sets and/or methodologies. "Treemometers" have been most definitively trashed, but tree ring data is used repeatedly throughout the literature."

Here's something I absolutely can't figure out- everybody throws out the Briffa tree stump data after 1960 because it doesn't match up with instrument data. That being the case- what gives anybody faith in ANY of the tree ring data? Or contrary to that, any faith in the station instrument data. Assumedly something is off in one of them. I absolutely cannot understand how these folks can bring themselves to use the tree ring data for the past but not for the last 50 years with a straight face.

Quick hit: New Zealand's ClimateGate.

Chris opines:

Even the infamous CRU has a substantial amount of data archived

Umm ... huh?

SCIENTISTS at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based.

It means that other academics are not able to check basic calculations said to show a long-term rise in temperature over the past 150 years.

The UEA’s Climatic Research Unit (CRU) was forced to reveal the loss following requests for the data under Freedom of Information legislation.

The data were gathered from weather stations around the world and then adjusted to take account of variables in the way they were collected. The revised figures were kept, but the originals — stored on paper and magnetic tape — were dumped to save space when the CRU moved to a new building.

I guess "has a substantial amount of data archived" means "well, the data's not actually archived, and we had to massage it quite a bit." But yeah, pretty much what you said, Chris....

Jesus. Do you really have that low an opinion of anyone besides yourself? Really?

I'm probably going to regret this, but there are some things Nukem says I'm curious about, so...

The article titled: Description of the "TRIFFID" Dynamic Global Vegetation Model, by Peter M. Cox, appears to be a study of how vegetation reacts to climatic forcing. Not about the impact of external forcing on climate. Now, there is discussion about the interaction of vegetation coverage on climate, but this is not nearly the clear-cut climate model and supporting data that "proves" AGW. Way to go Chris.

Out of curiosity, where did I link to this? I don't remember linking to it, and can't find it through a cursory search of my comments on this thread. And, assuming I did link to it, where did I say or suggest that it "proves" AGW? Just looking for some cites here.

Chris mentions the thousands of research articles that "confirm" AGW.

Similar deal - I don't think I ever mentioned "thousands of research articles." Where did I say this, exactly?

For example: CRU is, in fact, in almost complete control of committee placements for the IPCC.

Cite, please?

He and Ross McKitrick pretty thoroughly demolish the Hockey Stick. And the magical mystery tour that AGW-alarmists attempt to take us on in "refuting" M&M is nothing short of mendacious.

This includes the National Research Council report that's been linked to ad nauseum on this thread, I assume? Do you have an actual argument why this is the case and why the NRC report is flawed, or is this just an unsourced assertion on your part?

Chris is basically full of crap. His basic MO is to throw a bunch of bullshit at us, in the hopes we either won't follow the links, or won't have enough background to understand his deviousness.

Well that's... certainly an interpretation of what I've written, I'll give you that. I think in general I'll let people judge between my arguments backed up with "bullshit links" vs. your largely unsourced arguments, and leave it at that. But I'm still curious about where I said these things you said I did, so please let me know about that, ok?

Have a good one, Nukem.

Leave a comment

Here are some quick tips for adding simple Textile formatting to your comments, though you can also use proper HTML tags:

*This* puts text in bold.

_This_ puts text in italics.

bq. This "bq." at the beginning of a paragraph, flush with the left hand side and with a space after it, is the code to indent one paragraph of text as a block quote.

To add a live URL, "Text to display":http://windsofchange.net/ (no spaces between) will show up as Text to display. Always use this for links - otherwise you will screw up the columns on our main blog page.




Recent Comments
  • TM Lutas: Jobs' formula was simple enough. Passionately care about your users, read more
  • sabinesgreenp.myopenid.com: Just seeing the green community in action makes me confident read more
  • Glen Wishard: Jobs was on the losing end of competition many times, read more
  • Chris M: Thanks for the great post, Joe ... linked it on read more
  • Joe Katzman: Collect them all! Though the French would be upset about read more
  • Glen Wishard: Now all the Saudis need is a division's worth of read more
  • mark buehner: Its one thing to accept the Iranians as an ally read more
  • J Aguilar: Saudis were around here (Spain) a year ago trying the read more
  • Fred: Good point, brutality didn't work terribly well for the Russians read more
  • mark buehner: Certainly plausible but there are plenty of examples of that read more
  • Fred: They have no need to project power but have the read more
  • mark buehner: Good stuff here. The only caveat is that a nuclear read more
  • Ian C.: OK... Here's the problem. Perceived relevance. When it was 'Weapons read more
  • Marcus Vitruvius: Chris, If there were some way to do all these read more
  • Chris M: Marcus Vitruvius, I'm surprised by your comments. You're quite right, read more
The Winds Crew
Town Founder: Left-Hand Man: Other Winds Marshals
  • 'AMac', aka. Marshal Festus (AMac@...)
  • Robin "Straight Shooter" Burk
  • 'Cicero', aka. The Quiet Man (cicero@...)
  • David Blue (david.blue@...)
  • 'Lewy14', aka. Marshal Leroy (lewy14@...)
  • 'Nortius Maximus', aka. Big Tuna (nortius.maximus@...)
Other Regulars Semi-Active: Posting Affiliates Emeritus:
Winds Blogroll
Author Archives
Categories
Powered by Movable Type 4.23-en