Friday, November 14, 2008

How to work with real statistics

Lorne Gunter called on skeptics to unite. He did so in the National Post. His story was about scientists who don’t warm up to “the orthodoxy on global warming”. What a shame that but few got this call because it came on Monday, October 20, 2008. The timing couldn’t have been worse. It was another Monday when Wall Street and Bay Street watchers saw stock indices move straight south. Global warming isn’t of as much concern as are shrinking stock portfolios. It may explain why Lorne’s tale was told on a Monday. Sandra Rubin’s story, too, ran on a Monday. NP’s head honchos run their own stories mostly in weekend editions. NP’s very first edition was printed on October 27, 1998. At that time, it was Lord Black’s pride and joy. At this time, Lord Black is doing time and NP’s kingpins are still timing things their own way.

Lorne need not have urged skeptics to unite since they did so long ago. Skeptics do hold a dim view of pseudo scientists who play games with scientific integrity. I may well have been a born skeptic. I was taught more than I could grasp about heaven and hell from a pulpit in a Dutch village. Nowadays I teach how to test for spatial dependence in sampling units and sample spaces. Stanford’s Journel taught in 1992 that spatial dependence between measured values may be assumed. I never thought much of Journel’s thinking. Neither did JMG’s Editor. All I thought about at that time was to rid the world of Matheron’s junk statistics. Come hell or high water! And I still do!

The National Post brought to light on November 7th that President-Elect Barrack Obama is set to “Stop global warming”. It brought back that off the wall “Stop continental drift” slogan. Geologists slowed down continental drift by calling it plate tectonics. Plates are still moving, and earthquakes, magma flows and tsunamis are tagging along. The National Post on November 10 claimed that climate change, too, is on some kind of yes-we-can list. Surely, geoscientists should study climate change. What the study of global warming has done so far is set the stage for a constant belief bias.

Lorne’s story about skeptics and global warming came about because of the work of Professor Dr John R Christie. More than 300,000 daily temperature readings around the globe with NASA’s eight weather satellites over 30 years gave Christy and his coauthor a massive data set to work with. It was marked “Lower Troposphere Global Temperature: 1979-2008.” The authors had drawn a trend line thru a see-saw plot. It was the shape of this trend line that piqued my interest. What I wanted to do was test for spatial dependence between measured values and determine where orderliness in our own sample space of 30 years dissipates into randomness. So I asked Lorne and he did sent me the whole set that underpins the plot in his story!

The first step in the statistical analysis is to verify spatial dependence between observed temperatures in this sample space of time by applying Fisher’s F-test to the variance of the set and the first variance of the ordered set.

The observed value of F=6.27 exceeds the tabulated value of F0.001;df;dfo=1.32 at 99.9% probability by a margin of magnitude. Hence, monthly temperatures display an extraordinary high degree of spatial dependence. The probability that this inference is false is much less than 0.1%.

The second step is to verify whether or not the weighted average difference of 0.063 centigrade is statistically identical to zero. Since the first set and the last one have different degrees of freedom than intermediate sets, Student’s t-test is applied with a month-weighted average variance. Such weighted variances are called pooled variances in applied statistics.

The observed value of t=4.245 exceeds the tabulated value of t0.001;dfo=3.674. Hence, the probability is less than 0.1% that this weighted average difference of 0.063 centigrade is statistically identical to zero. Alternatively, this probability of 99.9% points to a statistically significant but small change of 0.063 centigrade during this 30-year period. Detection limits that take into account Type I risk only and the combined Type I and II risks are of critical importance in risk analysis and control. In this case, the Type I risk is ±0.031 centigrade, and the combined Type I risk and Type II risk is ±0.056 centigrade.

The third step is to verify whether or not the variances of ordered temperatures in centigrade constitute a homogeneous set.

Bartlett’s chi square test shows that the observed χ2-value of 22.979 falls between 42.557 at 5% probability and 17.708 at 95% probability. Hence, the set of variances for this 30-year period is homogeneous.

Sir Ronald A Fisher was knighted in 1953 for his work with analysis of variance. Dr F P Agterberg fumbled the variance of his distance-weighted average point grade in 1970 and in 1974. NASA started to measure Lower Troposphere Temperatures in 1979. I showed how to test for spatial dependence between metal grades in ordered sets for the first time in 1985. So why would any geoscientist assume spatial dependence between measured values in ordered sets? Agterberg is the President of the International Association for Mathematical Geosciences. He should explain why his distance-weighted average point grade does not have a variance.