Tuesday, February 24, 2009

APCOM calling

One more acronym to remember! APCOM stands for Application of Computers and Operations Research in the Mineral Industry. It adds up and it stuck. My son knows a lot about EMF and I know a little about metrology in the mineral industry. We added it up and posted an abstract for our APCOM paper. We did so before the first deadline in 2008 came and went. APCOM is still calling for papers. Another deadline passed on February 15, 2009. Where have all those abstracts gone? Popular myth has it that abstracts always swamp program chairs. APCOM is to be held at Vancouver, British Columbia, Canada, from October 6 to 9, 2009. It 's high time to complete the program.

APCOM’s preliminary program brings up The Future of Mining and Technology’s Role. It doesn’t add up to another acronym but does have a futuristic ring to it. The first day is set aside for Resource Identification Estimation and Planning. That's a load of cool stuff. And it’s comes to my home turf in Vancouver, British Columbia. I would like to show resource planners how to estimate metal contents and grades of reserves and resources. It fits APCOM’s preliminary program like a silk glove. We wrote about it in 1991. We want to talk a bit more about it in 2009.

But wait a pixel-picking second! Where would our paper fit in? Geostatistics 1 on the first day? And Geostatistics 2 on the same day? Looks like Geostatistics for the Next Century all over again. That was quite a krige-and-smooth-fest. It took place at Montreal in 1993 with Chairperson Dimitrakopoulos in charge. At that time he was all wrapped up at the cutting edge of conditional simulation with voodoo variances. What I wanted to do was talk about The Properties of Variances. Nowadays, Dimitrakopoulos is Editor-in-Chief, Journal for Mathematical Geosciences. He is still teaching McGill’s students all he knows about conditional simulation. Despite the fact that signs of change to come are posted on IAMG’s website.

In fact, IAMG in 2009 is promoting mathematics, statistics and informatics. I couldn’t have made it up. No longer does it stand for International Association for Mathematical Geology but for International Association for Mathematical Geosciences. Its mission is “…to promote, worldwide, the advancement of mathematics, statistics and informatics in the Geosciences”. How about that? Agterberg, IAMG’s Past President, has yet to explain why his distance-weighted average point grade doesn’t have a variance. He might one day be asked under oath to explain whether or not his distance-weighted average point grade has a variance.

The following formula gives Agterberg's distance-weighted average point grade of his set of five (5) point grades. Agterberg's real problem is that this formula converges on the Central Limit Theorem if all weighting factors converge on 1/n.

It’s a bit of a stretch for Agterberg and his associates “…to promote…” statistics but declare null and void David’s “…famous Central Limit Theorem”. It doesn't take a genius to prove that the next formula converges on David's famous one when each wi converges on 1/n. The question is then what Agterberg was thinking in 1970 and in 1974. Surely, it's about time he rights his wrong!

I’m a stickler for unbiased sampling practices and sound statistics. So, I’m not at all surprised that scores of geoscientists would rather work with real statistics than with surreal geostatistics. After all, statisticians test for spatial dependence, chart sampling variograms, and count degrees of freedom. No hanky panky with the kriging game. No ifs and buts about degrees of freedom! No functions without variances. All scientists and engineers on our little planet should make sound statistics a way of life.

Sunday, February 08, 2009

Junk statistics on Wikipedia

Wiki’s Kriging bugs me just as much today as did textbooks on geostatistics in the 1990s. What bugs me most of all is that Wiki’s keepers of Kriging didn’t give the set of measured values. That brought back many bad memories of Matheron’s magnum opus. His 1954 Formule des Minerais Connexes is peppered with formulas and symbols, gives but few a derived statistics, and no sets of measured values. This paper is posted as Note géostatistique No 1 but is itself marked Note Statistique No 1. Somebody played a silly game in predating the birthday of Matheron’s new science of geostatistics. Wiki's link to Matheron's seminal work went dead on December 12, 2008. Click Centre de Geosciences, and go to Ressources Documentaires & Logiciels. Next, click Bibliothèque Géostat (en ligne) and take a long look at Matheron’s past. This link is still hot, and I’m tickled pink. I’ve got to get this new link on some of my old blogs.

Wiki's Kriging keepers made me think of Matheron's statistically challenged disciples, and of all their tangled thoughts. Why do formulas and symbols run rampant where sets of measured values are as scarce as hen’s teeth? Little odds and ends of geostat speak such as “…a system of linear equations which is obtained by assuming that ƒ is a sample path of a random process F(x)…” make me cringe. Sounds a bit like Matheron’s take on Brownian motion. Why did degrees of freedom fail to inspire the wardens of Wiki’s Kriging? Of course, it would explain why they just keep on kriging for life!

I had asked for but never got the set of measured values that underpins Figure 1. So, I derived the same measured values in scale units. Fisher’s F-test proved that the ordered set of scale units does not display a significant degree of spatial dependence. The guardians of Wiki’s Kriging pointed out, “From the geological point of view, the practice of kriging is based on assuming continued mineralization between measured values”. What a way to practice kriging! Stanford’s Journel espoused the same sort of assumed nonsense in 1992. I never took him serious but he may well have thought he was. I did what Journel didn't do in 1978. Several years before the Bre-X fraud I derived variances of density- and length-weighted average lead and silver grades of core samples. I worked with weighting factors since the set of measured values in Figure 1 is unevenly spaced.

I’m caught between real statistics and hardcore kriging. I interpolated by kriging between each pair of measured Y- and X-values. The spreadsheet template shows that the first pair gives a Y-value of 103.0 scale units and an X-value of 25.8 scale units, the second pair gives a Y-vale of 96.0 scale units and an X-value of 45.5 scale units, and so on for a set of seventeen (17) pairs. The following chart shows why interpolation by kriging does so much more with less. All it takes is to rig the rules of statistics.

False 95% confidence intervals

Now here’s the clincher. Fisher’s F-test cannot be applied to an ordered set of seventeen (17) values, each of which is either measured or kriged. The problem is sets of measured values do give degrees of freedom whereas kriged values give none. A simple rule of thumb is that measured values do give degrees of freedom whereas kriged values give nothing but headaches. Unless, of course, one grasps the irrefutable fact that each kriged estimate does have its own variance just as much as do central values such as arithmetic means and all sorts of weighted averages.


Listed above are 95% confidence limits for central values of nine (9) measured values only, and of seventeen (17) measured and kriged values. Interpolation by kriging between measured values seem to give a higher degree of precision do than measured values alone. It’s not so much that Krige knew how to work miracles with a few measured values but that Matheron’s disciples have rigged the rules of real statistics. To put it simply, a kriged estimate has its own variance since all functionally dependent values do. A reliable rule of thumb is kriged estimates give big problems whereas measured values give degrees of freedom.

One would expect Wiki’s Kriging squad to show how “95% confidence intervals” in Figure 1 were derived. Surely, the squad was joking when it put forward, “Assuming prior knowledge encapsulates how minerals co-occur as a function of space. Then, given an ordered set of measured grades, interpolation by kriging predicts mineral concentrations at unobserved points”. How about that? Sounds like Wikipedians live in Wonderland. Krige himself couldn’t have cooked up such drivel.

Thursday, January 29, 2009

Born krigers do more with less

Teaching statistics to born krigers takes a long time. What born krigers do best is fit curves. Think what happens when a curve is fitted through a set of measured values. Most of all there’s much pride and joy. A perfect curve is indeed a thing of beauty. Look at the Fourier transform of Wölfer annual sunspot counts from 1700 to 1987. Isn’t it as stunning as the original plot? It does put into perspective the power of mathematics when applied to an ordered set of measured values in our own sample space of time. A perfect fit is of less interest in my work than the statistics behind ordered sets of measured values. For example, I applied mathematical statistics to derive the statistics of Wölfer annual sunspot counts for the period from 1749 to 1924. I work with spreadsheet software because it is such a powerful tool to show and tell. Several Excel files are posted on my website under Statistics for geoscientists.


Wiki’s Kriging doesn’t test for spatial dependence between measured values in an ordered set. Wiki’s keepers of Krige’s grail didn’t even try. Here’s what they wrote about kriging, “The theory behind interpolation and extrapolation by Kriging was developed by the French mathematician Georges Matheron based on the Master's thesis of Daniel Gerhardus Krige.” It’s short and crisp but not to the point.


Krige’s 1951 Master thesis brings up ‘knowledge of mathematical statistics’, ‘careful statistical analysis’, ‘science of statistics’, ‘modern statistical basis’, ‘application of statistics’, and so on. It does read like a thesis on statistics, doesn’t it? Nowhere did Krige bring up ‘geostatistics’. A 2003 Tribute to Krige alluded to “…his pioneering work in the application of mathematical statistics…” The same tribute alluded to Krige’s 1952 paper in which he “introduced, inter alias, the basic geostatistical concepts of ‘support’, ‘spatial structure’, ‘selective mining units’, and ‘grade-tonnage curves’. Did it take Krige one year and a bit of inter alias to switch from real statistics to a pinch of between-the-lines geostatistics? Not quite! He was a committed geostatistician when he wrote the Preface to David 1977 Geostatistical Ore Reserve Estimation. But when did Krige really take to kriging?


Matheron’s Note Statistique No 1 saw the light of day in North Africa on November 25, 1954. He coined the first krige-inspired eponym in his 1960 Krigeage d’un panneau rectangulaire par sa périphérie. Matheron didn’t refer to Krige’s 1951 Master thesis. Neither did he much refer to anyone’s work but his own. In those early days Matheron himself dawdled between statistics and geostatistics. But he was not much of a statistician even though he thought he was one.


It makes sense to compare Wiki’s Kriging with Krige’s teachings. Look at Figure 1 in Wiki’s Kriging. The graph didn’t irk me quite as much as did the confidence intervals for measured values. Once upon a time I tried to get the set of measured values that underpin Figure 1 but its caretaker(s?) didn’t respond. So, I waited until it was time to take a stand against junk statistics on Wikipedia.


Figure 1


One-dimensional data interpolation by kriging, with confidence intervals.

Squares indicate the location of the data.

Kriging interpolation is in red.

Confidence intervals are in green.


I enlarged Figure 1 and measured X- and Y-coordinates for all points in mm. I tested for spatial dependence by applying Fisher’s F-test to the variance of the set and the first variance term for the ordered set. I applied weighting factors because of unevenly spaced measured values. That’s why degrees of freedom become irrational numbers.



Given that the observed value of F=var1(x)/var(x)=1,504/1,408=1.07 is below the tabulated value of F0.05;dfo;df=6.04, it follows that the ordered set of measured values does not display a significant degree of spatial dependence. Hence, measured values in the ordered set are randomly distributed within this sample space. Therefore, interpolation between measured values makes as much sense as extrapolation beyond the set. As a matter of fact, it does give junk statistics of the worst kind wherever and whenever randomness rules. I do not know how Wiki’s Kriging caretakers cooked up the confidence intervals in Figure 1. I applied plain vanilla statistics and plotted 95% confidence intervals in this bar graph.



The bars in this graph, unlike the measured values in Figure 1, are evenly spaced. I’ll show in another block that interpolation between measured values in an ordered set does indeed give the same sort of junk statistics as did Bre-X and the kriging game.

Tuesday, January 20, 2009

Working with Wikipedians

Wikipedia is a wonderful source of information for all of us while we are doing our time in this universe. Wiki is reliable as a rule and tries to do right when in doubt. For example, under Geostatistics Wiki points out, “This article is in need of attention from an expert on the subject. WikiProject Geography or the Geography Portal may be able to help recruit one”. No kidding! Wiki’s expert would have to be some kind of jack-of- all-sciences. So many disciplines do have a role to play in geography.

Geologists and mining engineers got stuck with geostatistics when Matheron goofed but thought he had dug up a new science. They were taught not work with the Central Limit Theorem and to infer ore between widely spaced boreholes. To infer ore between step-out boreholes at a spacing of 200-m worked well indeed in the Bre-X case. On the other hand, to infer spatial dependence between closely spaced pixels makes sense. When I tested for spatial dependence between gold grades of ordered rounds in a drift, Journel called me "too encumbered" with Fisher’s statistics. It’s not surprising then that geoscientists at Stanford are taught to assume, krige and smooth voodoo variances. Geoscientists with a passion for order tend to do curve-fitting. Too many are led to believe that geostatistics is good for geoscientists. I know that geoscientists would enjoy working with real statistics just as much as Sir Ronald A Fisher once did.

I tried to add applied statistics to Wiki’s Geostatistics when it was still called Kriging. I did so when I was a new Wikipedian in 2005. I knew then that geostatistics is an invalid variant of applied statistics. My son and I had known why since the early 1990s. What I didn’t know in 2005 is who stripped the variance off the distance-weighted average. But I do know now who did and when! What I do not know is why. I’ll continue to explain my case against geostatistics in concise terms and with significant symbols. I do so not only as a member of several ISO Technical Committees but also as a blogger, as a webmaster, and, last but not least, as a Wikipedian.

Most Wikipedians have a strong need to leave a better informed world than we found. I’m no exception. I hold an edge in always having worked with applied statistics and grasped Visman’s sampling theory and practice. I know that geostatistics converted Bre-X’s bogus grades and Busang’s barren rock into a massive phantom gold resource. What I also know is that bogus assays for three to five salted boreholes would have been enough to nip this mind-boggling fraud in the bud. The world’s mining industry doesn’t want to know is what I would have done!

Neither does Pierre-Jean Lafleur want to know. He is a Professional Engineer and a reserve and resource expert with Watts, Griffis, and McOuat Limited. He doesn’t believe I called the Bre-X fraud several months before the boss salter vanished. Lafleur wrote, “The information he provides is unclear, and most likely untrue”. So he wiped it off Wiki’s Bre-X Minerals. Neither may he believe it was not I who put my name on that Wiki subject. But what I did do when my name came up with the wrong context was add the facts and a few links to subjects such as spatial dependence and sampling variogram.

Lafleur deserves some praise because he doesn’t work under a nom-de-plume. Too many Wikipedians work anonymously. When scientists and engineers want to be taken seriously on Wikipedia they should stand up and be counted. Wikipedia should not allow Wikipedians who hide behind pseudonyms to delete indisputable scientific facts. Examples in my discipline of sampling and statistics are the Central Limit Theorem, functional dependence, spatial dependence and degrees of freedom.

Look and see which stats derive from Matheron’s Formule des Minerais Connexes. What a pity that the seminal work of the Creator of Geostatistics and the Founder of Spatial Statistics is no longer posted on the web. In fact, Matheron was a self-made wizard of odd statistics. Here's the link to Matheron's correction of his very first paper. Enter a different number of core samples and see how it impacts 95% confidence limits. Real statistics does all kind of cool things!

Professional engineers and geoscientists claim to be guided by codes of ethics that protect the public at large. Provincial securities commissions in Canada employ reserve and resource experts to set the rules. But should foxes run henhouses? That’s a good-enough reason why a National Securities and Exchange Commission should turn provincial fiefdoms into branch offices. Reserve and resource experts in branch offices should then be asked to testify under oath and explain why the Central Limit Theorem and degrees of freedom are null and void in geostatistics.

Working with applied statistics is fun. And it’s kind of cool for our planet! Wikipedians should read what the International Association for Standardization is all about. ISO may violate the odd copyright, and ignores priority once in a while. And the UN is not perfect either. Only Wikipedia can bring scientific integrity to the world.

Wednesday, December 31, 2008

Agterberg's way

Here’s what Agterberg wrote to me, “It seems that you are an iconoclast with respect to spatial statistics including kriging.” He did so in his reply to my email of October 7, 2004, on the subject of The Silence of the Pundits. That’s not quite what I had written to him. I didn’t bring up spatial statistics or kriging. It seemed as if Agterberg’s tribute to Matheron had become his new reality. All I had asked were questions about the distance-weighted average. I didn’t know in 2004 that Agterberg himself had derived this distance-weighted average point grade first in his 1970 Autocorrelation Functions in Geology and once more in his 1974 Geomathematics. What kept me spellbound in this Millennium was Matheron’s mind-numbing opus after it was posted on the website of the Centre de Géosciences. Since December 12, 2008, all I get to look at is “Not found.” I was used to Matheron’s prose and symbols but did miss his primary data. I wish his collected works were posted for posterity. It is such stunning stuff.


Agterberg brought up a friend of mine with similar criticisms who had “orally presented his views at IAMG meetings.” Agterberg thought I might wish to do the same. Good grief! What I do is put my thoughts in writing. I did so with The Properties of Variances in 1993. I wanted to bring the properties of variances within the grasp of geostatistical thinkers. Many had gathered at McGill to celebrate Geostatistics for the Next Century. It sounded somewhat premature but geostatistics was growing in leaps and bounds in those heady days. The properties of real variances were rather late in coming and the Bre-X fraud was just around the corner. As luck would have it, the properties of variances didn’t quite suit the tribute to David’s work with its infinite sets of simulated values and zero pseudo variances. That sort of science fiction still underpins McGill’s curriculum for budding geoscientists. McGill University is a source of goofy geosciences.


Philip and Watson’s Matheronian Geostatistics: Quo Vadis? (MG, Vol 18, No 1, 1986) made Matheron fit to be tied up. His rebuttal took the form of a Letter to the Editor (MG, Vol 18, No 5) on the subject of Philipian/Watsonian High (Flying) Philosophy. Agterberg’s way is oral criticisms but I really liked Matheron’s written rebuttal. On the other hand, Matheron’s temper tantrum driven tirade might have boggled the odd geostatistical mind. I wrote about voodoo statistics in the 1990s but it failed to trigger another mind numbing tirade.


Matheron was called the Founder of Spatial Statistics and the Creator of Geostatistics. Why did his ramblings merit twin epitaphs? The more so since Berry and Marble’s 1968 Spatial Analysis, a Reader in Statistical Geography, makes no mention of Matheron’s work. Chapter 8 Fourier Analysis in Geology in Section IV Analysis of Spatial Distributions refers to Agterberg’s Methods of Trend-Surface Analysis. Agterberg talked about it at a 1964 symposium with Applications of Statistics in its lengthy title. Just the same, Matheron did dismiss trend surface analysis at the 1970 geostatistics colloquium. Why did the masterminds not see eye-to-eye on spatial statistics when Matheron brought his new science to the USA?


All that gibberish troubled me even more when I read Agterberg’s response to my questions of October 11, 2004. On September 23, 2004, I had posed the same questions to the Councilors of the International Association for Mathematical Geology, and to the Editor and his Associate and Assistant Editors of the Journal for Mathematical Geology.


Who lost the variance of a single distance-weighted average?

Who found the variance of a set of distance-weighted averages?


Only one Assistant Editor responded by pondering, “If geostatistics is not furthering a certain problem, a different type of mathematics may solve it.” Now there’s one partially open JMG mind at work! It didn’t tempt me into giving oral criticisms at any IAMG meeting.


Here’s what I wrote on October 12th in response to Agterberg’s Aberdeen message of October 11, 2004. “I just want to know when and on whose watch the variance of the single distance-weighted average vanished, and when and under whose tutelage the kriging variance and covariance of a set of kriged estimates became the cornerstones of geostatistics, spatial statistics, kriging, smoothing, or any other popular computation that violates the requirement of functional independence and the concept of degrees of freedom”. His way was not to respond.

Agterberg had failed to derive the variance of his distance-weighted average point grade first in 1970 and again in 1974. What he did do was make a sham of scientific integrity when he was IAMG’s President. He did call it the International Association of Mathematical Geosciences. Agterberg’s way was to stay silent. It’s the wrong way in science. The right way would be to revise Geomathematics.