Are indicators yesterday's news?

This is Sydney, so here is some scientific Sydneyana (Figure 1): Harry Recher's famous correlation between two indicators, bird species diversity and foliage height diversity (Recher 1969). Harry, of course, was the first ecologist at the Australian Museum, and this figure was considered terribly important at the time. Even now it shows a certain naive charm. But we must be gentle, and remember that it was 1969, that diversity was a hot number in theoretical ecology circles, and that many of us thought we were closing in on the answers to the big questions of ecology.

Figure 1. Two indicators of ecosystem structure (after Recher 1969)

That we were wrong is due in part to hubris, the constant companion of all theoretically inclined scientists, but it is also due in part to a belief that we could substitute measurement for understanding. That we could lazily capture understanding the way we capture images on a photograph, by clicking a button - in this case, the button on one of the new programmable calculators that were just then coming onto the market.

Now I am not going to propose that we lay all our problems with indicators at the feet of Hewlett Packard and Texas Instruments, because they produced calculators that made it easy to calculate diversity indices. But I am going to say that they made it easy for us to play with indicators, to fall for their seductive charms, to create a sad Cartesian parody: Indico, ergo sum.

But first I need to establish some context. I am assuming that we are all in the same game here, that we are trying to try to get a handle on complex systems, such as ecosystems and economies, that we are trying to unravel their complexity so that we can further some agenda that includes any or all of that science triptych: explanation, prediction and control (Bradbury et al. 1983). Now some people, perhaps many in this audience, believe that we can do this by creating some variate, usually one-dimensional, which tells us something useful about the system, while others, myself included, think that this is a dumb thing to do.

Harry Recher's indicators illustrate my point admirably. Bird species diversity is trying to say something about the relative abundance of the different species that make up a community. It is also trying to make some inference about the structure of that community, and, drawing an even longer bow, about its dynamics. And it is trying to do this in a compact, tidy way. Likewise, foliage height diversity is trying to say something compact yet profound about the nature of the physical structure of the forest. These are quintessential indicators.

But that was, as I said, the sixties. Let's take a closer look at some of today's indicators. Let's dissect some to see if they really are as dumb as I claim. They are not hard to find in today's indicator addled society. Nor is it hard to find some really stupid ones. It is not even hard to find some really stupid ones that many people think are terrifically important. Take GDP , for instance.

Why pick on GDP when there are plenty of other really stupid indicators to shred, and GDP has at least the virtue of internal consistency? Well, it is because GDP is the mother of all indicators. It has been around in one form or another since Sir William Petty first attempted to estimate England's national income in 1665. And, importantly, in its present form, it is constructed so as to breathe life into a Keynesian approach to the economy. It is the key indicator for a Keynesian system whose macroeconomic flows have been reduced to production, income, consumption, investment and saving (Castles 1990). So built, GDP not only purports to capture something important about the economy, but also shores up a particular model of the economy by making it difficult to view it in any other way.

Now what exactly does GDP indicate? You might be naive enough to think that GDP - Gross Domestic Product - should indicate just how much we produced somewhere or sometime. But you only have to scratch GDP and it bleeds - big time.

For instance, my mum, like most mums of her generation, stayed home. She didn't work after she was married. That is, she didn't work in what economists call the 'workforce'. But whenever I saw her, she was working her butt off, producing stuff - putting up jams and pickles, making frocks and shirts, knitting jumpers, cooking dinner, growing vegetables, cleaning house. But is all this product, this production, added into the GDP? No way. Because she didn't sell it, I suppose, so it wasn't part of the fiction we call the economy.

So we have invented an indicator that ignores mothers!

Mothers aren't the only part of society to be ignored by this great little index. It also ignores people who help each other. Think of all the voluntary work done in this country. All the P&Fs that keep the schools afloat; all the Smith Families and Salvos that help those who are down on their luck in this, the lucky country; all those who give their time to read to inmates of the old folks' homes; or act as Scout leaders or Life Savers. Is all that sweat and tears, all that production counted, does it contribute to the big number? 'Fraid not. No money changes hands, so all this effort doesn't really exist as far as GDP is concerned.

So we now have an index that ignores the young, the old and the poor. Not to mention the Scouts and Life Savers, for chrissake.

Has any old lady ever refused a boy scout's offer to help her cross the road because he doesn't really exist? Has a drowning tourist ever rebuffed the beltman hauling her out of a rip because he is imaginary?

But wait, there's more. Even if we accept the narrowest definitions, and its attendant circularity, of society and its products to include only that which is produced and exchanged for money, GDP still gets the booby prize. You only have to look around you. One of the major agricultural crops in this country is Indian hemp, Cannabis sativa, marijuana. It has many uses, including fibre and stockfeed, and some people, I am told, even smoke its dried leaves. It is the backbone of the economy of many country towns, and supports an extensive network of wholesalers, distributors and retailers. There are suburbs in my home town, Canberra, with huge houses, known to the locals as 'grass castles'. It is the quintessential cash industry. Everything, I am told, is cash on the nail.

Now I am spelling this out in tedious detail, even if it is old news to many in this audience, because this is hot news for the people who build up the GDP index. They missed it. Or they forgot to put it in, or they didn't really think that, just because this trade happens to be illegal, that it should be included.

So now we have an indicator that ignores farmers.

I could go on and discuss our important import-export trade in narcotics. But I think I have made my point.

And the point is this: when we do a reality check on GDP, that is when we look around us, we notice that GDP is a sad little index, a deluded index, of no use to anyone who wants to understand the real world. Its only use is a self-serving one: it provides a tor for the great game of pretending that general equilibrium models of the economy actually mean something.

More than that, any indicator that ignores farmers, old ladies, scouts, mothers and life savers is positively unAustralian.

Now to prove I am not unduly biased against the social sciences, I want to look at a really stupid indicator from the natural sciences.

If GDP is the mother of all indicators, then the WQI, the Wilderness Quality Index, is the mother of all surrogates (Lesslie & Maslen 1995). It tries to act as a token for naturalness, which in turn tries to act as a token for something like ecosystem function itself. It gets into this bind because the idea of wilderness muddies the distinction between science and ideology. It confounds the scientific notion of ecosystem function with the 'noble savage' idea of civilisation's loss of communion with nature. In so doing, it imputes qualities to wilderness which border on the mystical. It has become a token of naturalness as if modern man were somehow not natural, not a natural product of natural selection, and as embedded in the ecosystem as everything else.

The idea of wilderness therefore takes the place of the much harder, and more scientific, ideas of ecosystem, and the indication of wilderness takes the place of the measurement of ecosystem structure and function.

Even as a surrogate it doesn't measure up. Because it is constructed in a most bizarre way. The WQI is a metaindicator. It is itself built up from four separate indicators: remoteness from settlement; remoteness from access; apparent naturalness and biophysical naturalness. The first three of these purport to measure remoteness and the last measures naturalness, which we are assured together make up the 'two essential attributes' of wilderness (Lesslie & Maslen 1995).

Now watch carefully while I play a pea and thimble game here. For any point in Australia, I will now construct three remoteness indicators and a naturalness indicator. If it is, say, the remoteness from settlement indicator, I will first measure the distance from the nearest settlement. A nice hard continuous measurement on the interval scale. Next I will weight this measurement to account for the type of settlement. (I have previously measured all settlements on an ordinal scale and assigned them into one of 4 ranks reflecting their size.) In so doing, I convert my interval measurement to an ordinal measurement. Lastly I do some tidying up to produce a standardised measurement.

The same sort of gobbledegook applies to each of the other indicators, which are then recorded as 'continuous floating point variables'. To get the final Wilderness Quality Index, it is now simply a matter of adding up the four final numbers.

Now did you spot the deliberate mistake? It's easy, really. As soon as we moved from measuring the distances to weighting them, we moved from interval to ordinal scales of measurement. Now this shift is intransitive, you can't go back. And the properties of the ordinal scale are not isomorphic to the numerical system known as arithmetic. To put this in less highfalutin' terms, you can't add this stuff up. To put it bluntly, the WQI is rubbish.

Sadly, this has not stopped the WQI being advocated for quite serious and important uses, such as designating wilderness areas and assessing wilderness loss.

Where did we go wrong?

You may think that that is the end of it. That we have established that indicators are stupid, and that's that. But for something as awful as indicators, we must not stop there. We must not only pillory them senseless and make them a laughing stock - two essential functions of any enlightened system of justice - we must also investigate their origins to see if any other silly ideas are lurking around. Witch hunts are after all the third, and most enjoyable, part of the majesty of the law.

So let's heat up our implements, dust off our thumbscrews, and bring our two prime suspects, ecology and economics, into our star chamber for a little fun.

Too easy. You can see ecology breaking down at the very sight of its inquisitors. It contritely confesses that it didn't think of indicators by itself. It got them, it says, from economics. Economics, sensing the proximity of the brazier and its irons aheating, quickly says 'statistics, we got it from statistics'. Then the whole sorry story comes blurting out.

We find that the origins of indicators are indeed in statistics, whose own origins are not at all scientific, but in the need to provide information to assist in the administration of the state, hence the name. Sure there are shoals of scientific indicators about, or at least indicators about what you and I would normally think of as scientific subjects - things such as the distribution and abundance of hydrogen ions in the regolith, which we call soil physics; or the distribution and abundance of animals such as koalas and human beings, only one of which we call ecology, while the other we quaintly call 'the census'. Most of these indicators are johnny-come-latelys. The real indicator action, for the last century or so, has been next door in economics.

Back in the nineteenth century, when the modern nation state was just getting up a head of steam (so to speak), and modern economics was emerging from whatever primal ooze, the fathers of economics looked across the fence to the house of science and saw what fun the physicists were having. Remember, the physicists at that time were extremely smug, having crowned physics 'queen of the sciences', and having declared that science was basically done, what with Newton and calculus and all. All that was needed was a little tidying up here and there.

Now the proto-economists looked at physics and fell for it, fell for the whole ball of wax. They cloned the eternal equilibrial physics of Newton into economics and produced what they called the 'general equilibrial theory' which, with a few parameter changes, looks like, is, Newtonian mechanics. You can see why they did: nineteenth century physics is highly idealised, tidy, internally self-referencing, and satisfyingly complex. It is complexity, Goldilocks style: not too complex, just enough. Brian Toohey (1994) - another Sydneysider - has unravelled this wonderful tale in his excellent book 'Tumbling Dice'.

The catch was the economists jumped too soon, they pinched the wrong paradigm. No sooner had they appropriated the ideas of physics, physics and the rest of science went into the meltdown that began with Darwin, continued with Einstein and is still with us today. Its legacy is a new understanding of the world which stresses complexity over simplicity, non-linearity over linearity, chaos over equilibrium, holism over reductionism. In fact it spits in the face of all the certainties that both renaissance science and modern economics are built on.

But back to indicators. We can imagine them emerging painlessly, effortlessly when a simplistic, linear, equilibrial, reductionist paradigm is joined with the very human lust for lists - the desire to organise and codify, the desire to bureaucratise the world. And we can imagine the whole process catalysed by the desire of bureaucrats to find new ways to employ themselves. Never underestimate this. If you ever have doubt, peruse the shelves in the OECD section of the library.

Now how did they cross over to science? The same human reasons: they were easy to measure, providing the semblance of activity; they offered a tidy codification of ecological processes, where before there had been none; and they had the distinct advantage of being already invented, it being always easier to copy ideas than to think for oneself.

Inquisitions are moral activities, as I am sure you appreciate. And this one is no different. It even provides a moral apiece for our two misguided souls. The first is for economists: Pick your paradigm carefully before you shift. And the second is for scientists: You reap what you sow. By doing the economists a bad turn and not stopping this Newtonian nonsense, we have now been paid back in spades.

Why did we go wrong?

Now none of this is to impugn the motives of those involved. People can work honourably and hard and still get it wrong. In medieval times, honourable, hardworking people spend lifetimes trying to estimate the numbers of angels that would fit on a pinhead, as an indicator of the mystery of faith. Science is not about our motives, or our honour. Nor is it democratic. What is scientifically right does not depend in any way on how many people believe it to be right, nor on whether the idea is popular or not.

And so it is with indicators. We need to understand why they are wrong just as much as we have understood where they are wrong. Indicators, despite their popularity, are the consequence of an approach to understanding the complexity of the world which is fundamentally and fatally flawed.

They are wrong because they are a pathological corruption of the reductionist approach to science. They are the reductio ad absurdum of reductionism. They are voodoo science.

They are wrong because they try to take reductionism, itself a suspect method when dealing with complex systems such as ecosystems and economies, to a new and pathological depth. They seek to reduce, to collapse, the dimensionality of some description of a complex system, such as the troposphere, the savannah or the Australian economy, while retaining our understanding of that phenomenon. They are trying to make one-dimensional models of systems with hundreds or thousands of dimensions which yet retain some explanatory power. Like throwing shadow rabbits on a wall, they can never capture reality. They remain caricatures.

We got into this mess through a mixture of grasping at straws and a desire to do something. Harry Recher's diversity indexes can be seen sympathetically in this light: here was something we could measure, here was something waiting for a programmable calculator, here was something that gave us the satisfaction of a number instead of the awfulness, the messiness of the narrative natural histories of the time. Here was something that seemed scientific because it seemed quantitative.

Have I been too hard? I will admit to some rhetoric here. But provoked rhetoric, provoked by much silliness with indicators. I will admit to sketching my argument in black and white, ignoring any shades of grey. I will even concede that there may be - alright, then, there are - indicators which may be useful. Useful, but not swell. They are useful only in a very circumscribed sense: in the sense that they are created and used carefully. Created carefully to account for the critical dimensions of a system, and used carefully as only one of several simultaneous views of a system. In that circumscribed sense, I will concede that indicators may help rather than hinder understanding.

What should we do?

I have argued that indicators as an approach to complex systems are not only wrong, they are necessarily wrong. In abandoning them we should do something else that has at least a chance of being right.

The solution lies in radical action, in returning to the roots of ecology. And here I offer another piece of Sydneyana, this time from the fifties: Charles Birch's ecological canon (Table 1). Birch was, of course, the Challis Professor of Biology at that other university. With Andrewartha, he created a rigorous ecology that dominated the field for many years (Andrewartha & Birch 1954). It was flawed in its refusal to examine the emergence of ecosystem level dynamics from species interactions. But it contained more commonsense and insight into the ways in which organisms live their lives than all modern eco-indicators put together.

Now a Birchian view by itself is not much of an advance on narrative natural history. Indeed, if we apply the Birchian canon to each organism in a food web, as first described by Charles Elton (1927), we would get a stupendous, impossible description of an ecosystem, one which is compendious but not manipulable. And that of course is why this sort of approach was eclipsed by indicators at the evolving front of ecology.

This may have been a true reading of the situation during the sixties, when indicators were first in vogue in ecology: that we had to use indicators because a comprehensive model of the ecosystem was just too hard, it was beyond us. But beginning with systems ecology  (VanDyne 1969), and then continuing through the astonishing advances of the science of complex systems (Levins 1970; Wolfram 1984; Kaufman 1993; Gell-Mann 1994; Bossomaier & Green 1996) and its associated technologies of scientific visualisation (Kaufmann & Smarr 1993), the situation has been completely reversed. We now have a powerful 'back to the future' idea alive and kicking. It is now possible to realise Elton's dream of capturing the food web and its dynamics in their native richness. It is now possible to do justice to Birch's canon.

In thinking about this possibility some years ago, some colleagues and I tried to imagine just what would be needed to create such a science, a science useful to policy makers (Bradbury et al. 1985). We concluded that policy required a new coming together of theory, data, models and tools. Even then, we didn't give indicators a passing glance.

With the creation of the complex systems paradigm, we now have the theory. And in the context of the sustainable development of Australia, we now have some of the data, thanks to the evolution of the National Spatial Data Infrastructure, which provides the protocols and metadata for sharing, exchange and harmonisation of the nation's data assets. We have coherent models for many of the major sustainable development issues, such as soil erosion. We have tools which allow these models to be brought together in integrated frameworks under the complex systems paradigm, and which allow the interrogation and display of the models' dynamics through advanced scientific visualisation techniques.

Let me offer a couple of examples from my own agency to show what can now be done to provide policy makers with something more modern, more robust, more scientific and more understandable than the indicators we have been using hitherto.

In the first study, we are developing a continental model of the interaction of the climate system with the terrestrial ecosystems of Australia in order to better understand the impact of climate variability on those ecosystems, and to provide farmers with information to allow them to become more self-reliant. To do this we need to couple a comprehensive model of the atmosphere to a model of the terrestrial system, with an emphasis on the soils and vegetation. With our colleagues, Lance Leslie and Yaping Shao at this university's Centre for Advanced Numerical Computation in Engineering and Science, we are making great progress. Our work to date (Shao et al. 1996; Shao et al. in press) confirms that it is possible to build - and understand - a model of the whole continent at the kilometre scale which couples soil, vegetation and atmospheric models which together have hundreds of parameters.

In the second study, we are developing a regional model of some irrigated lands and their communities in the Murray-Darling Basin to provide these communities with the tools to explore possible futures, and to analyse and understand the consequences of decisions they may take about their ecosystems. This model couples hydrogeological models of groundwater dynamics with crop production models and socioeconomic models to create a system which captures the dynamics of the interactions between those communities and their environments. These numerically intensive spatial models operate with hundreds of parameters at realistic scales of hundreds of metres. The work has already established the feasibility of operating at these scales, and has produced useful analyses of the consequences of different land management practices (Veitch et al. 1993).

So it is time to come out of the cold, the cold shadow world of indicators. It is time to learn to approach the complexity, the richness of the world with theory, data, models and tools which honour that richness instead of subverting it, which acknowledge that complexity instead of denying it.

The real world is an exciting place, join it. And let yesterday's news be used for its time honoured role - wrapping the garbage.

References