The incursion dilemma: Is absence of evidence the same as evidence of absence?

Introduction

The issue before us - how to verify if a ship has really exchanged its ballast water - falls into a class of scientific problems that we must classify as profound. Not necessarily profoundly difficult, but profound, deep, complex. It is so for two very different and compounding reasons.

The first has to do with how we establish something to be true, to which the rational answer is, we use the scientific method. But I ask: Which scientific method? So for the first, we need to understand what constitutes evidence.

And that bears on the second, which has to do with the class of phenomena we are dealing with, which is, of course, living things. Living things, in their interactions with each other and their environment generate implicit complexity and subtlety that experience teaches us are irreducible. By that I mean we find it difficult to partition the problem up into the big bits that we should concentrate on and the little bits we can safely ignore. Complex adaptive systems, such as living things, do not take kindly to being treated as linear even to a first approximation - everything is a priori important. Thus the second problem turns on the nature of absence.

Evidence?

Accept for the moment that we can use a traditional hypothetico-deductive approach, that Sir Karl Popper looks benignly down on us from some mechanistic Cartesian heaven, and that we know a thing or two about null hypotheses.

Our problem, in logical positivist terms, is to decide, on the basis of the evidence, that a ship has actually exchanged its ballast water. In Popperian terms, this reduces to disproving the null hypothesis that the water in the ship's ballast tanks is the same as some other water. But what other water? Where the ship came from? The water it steamed through? The water in the port of entry?

Now the key thing to remember here is that we approach the truth progressively by eliminating or disproving those things which are not true. We never get to actually prove anything. So we need to be pretty careful in constructing our null hypothesis - the Aunt Sally we wish to knock over.

So let us erect our simplest null hypothesis: that there is no difference between the water in the ballast tanks and the water nearby.

We now make some measurements of the ship's water, and then compare those measurements against similar measurements of the nearby water - we test our null hypothesis.

Here is where the fun begins, because there are two types of errors that we can make when doing this test. We might reject the null hypothesis even though it is true. Were we to do this, we would commit what scientists, in their imaginative and colourful way, call a Type I error. On the other hand, we might accept the null hypothesis, when it is in fact false, committing that grave and grey sin - the Type II error. (With jargon this imaginative, it is no wonder that science courses are hard to fill in universities. By comparison, accountancy looks racy).

You have probably met Type I before. It has a night job as 'the confidence limit'. If you have ever come across statements like 'in a major study of the effects of smoking on health, scientists have discovered a significant difference in longevity between those who smoked and those who didn't', you have stumbled upon a confidence limit. Scientists usually talk about such differences being statistically significant within, say, the 95% confidence limits. All this means is that they think the differences that were measured would be real differences 95% of the time or 19 times out of 20. But the subtext of this is that they also mean that 5% of the time (or one time out of 20) we would see the difference and be fooled into thinking it was real, when in fact it just occurred by chance and was not a real difference at all. That is, 5% of the time we would reject the null hypothesis even though it is true. We would commit a Type I error.

By saying we accept a 95% confidence limit, we are really saying we accept a 5% Type I error rate.

Now you may not have come across Type II before, as it is a little more secretive. Nor is it simply the inverse of the Type I. It is harder to get a handle on because when we say we might accept a null hypothesis as true when it is in fact false, we are saying implicitly that there is some other null hypothesis that is in fact the true one. In some sense, we have erected the 'wrong' null hypothesis. So to strike an 'acceptable' rate for Type II errors, we need to know about the nature of the alternative hypothesis.

Our problem is, which alternative hypothesis? In the real world, as opposed to the constrained world of the experimental laboratory, the set of possible null hypotheses is large indeed.

Thus armed with our new knowledge of errors, let us return to the evidence.

Suppose our test shows that the water in the ballast tank is significantly different from the nearby water, at say the 95% confidence level. Can we infer from this that we may be reasonably confident that the ballast water is foreign? At first glance, we may say yes. And on closer inspection, we observe that there is only a small chance of us mistakenly believing that the ballast water is foreign when it is in fact local, that is that we have made a Type I error. We even may congratulate ourselves with the insight that making such an error is no bad thing, it being better to err on the side of caution.

But what of the Type II error, accepting the null hypothesis when it is false? Here, we have a situation where our test might find no significant difference between the ballast water and the local water. What can we now infer? We may infer that the ballast water is local. But what if the null hypothesis is really false? How likely is it that we have committed a Type II error? We cannot say, because we need to know something about the 'true' null hypothesis and we do not.

Thus we are in a situation where we may sometimes say with confidence that the ballast water is foreign when it really is, but we can never say with confidence that the ballast water is local. We can never confidently accept the null hypothesis.

You might think that the way out is to change the null hypothesis, but, I say again, which water do we choose? And even if we are happy with the choice, we now need to watch the Type I error which will always let a few cases slip through.

Our predicament, if we follow this evidentiary approach, is that we cannot be confident about our inferences, either because we do not know the size of the Type II error in the simplest case, or, in more subtle cases, because we do not know which more complex null hypothesis to erect.

Absence?

Let's leave Descartes' clockwork cosmos and pass instead to another, one perhaps governed by Arundhati Roy's 'God of Small Things'.

The paraphernalia of reductionist analysis survive in the Cartesian cosmos because of the sway of inverse laws in physics. Big things matter, and first order effects explain, and the rest can be safely stigmatised as noise or error. But in the other cosmos, biology with its exponential relationships, nonlinearities, and feedbacks, holds sway. Little things matter, multiple effects exist, and surprise and novelty characterise the system.

The ecologist who first enunciated 'The Tragedy of the Commons', Garrett Hardin, made the point eloquently in an essay[1] nearly 30 years ago on ecology as a subversive science:

There could hardly be a more trifling physical event than dropping a single bacterium onto a human pharynx ñ the mass of one body is 18 orders of magnitude greater than that of the other. But if the bacterium is a living pathogen, and the pharyngeal membrane is susceptible to the disease, the consequence may literally be of world-shaking importance. A man may sicken and die, starting an epidemic that weakens an empire and changes the course of history. All this is possible. It is also possible that nothing of historical importance happens. The difference between the two extremes is connected with nothing a physicist would notice. The initial bacterium weighs only about 7 x 10^-14 grams in either case; the significant differences are infinitely subtle.

While a physicist asks 'How big is it?' or 'How far away?', a biologist asks 'Does it increase exponentially?' or 'Is positive or negative feedback involved?'. To the biologist, nothing is, a priori, insignificant. There can be no Newtonian first approximation. The whole system must be considered.

In this world, we must not only ask different questions, we must also use a different scientific strategy. We need a strategy embedded in a modern understanding of complex adaptive systems.

One of the most pervasive linear assumptions (really, underpinning much of reductionist science so thoroughly that it is not even noticed) is that induction, the generation of general principles, is equivalent to the generalisation from observations, while deduction is its obverse, and equivalent to the creation of predictions of instances from general theory. CAS theory suggests otherwise and that induction/deduction are themselves neither simple nor a pigeon pair.

When CASs are involved (really always, since we are always involved), induction is an emergent process where ideas emerge, some living and some dying, from an interaction between all the CASs. This is much closer to the Einsteinian[2] spirit of pragmatism of the creative act, and its need to solve problems:

'Physics constitutes a logical system of thought which is in a state of evolution, whose basis cannot be distilled, as it were, from experience by an inductive method, but can only be arrived at by free invention. The justification (truth to content) of the system rests on the verification of the derived propositions by sense experiences, whereby the relations of the latter to the former can only be comprehended intuitively.'

or, again

'A theory can be tested by experience, but there is no way from experience to the setting up of a theory.'

Again deduction plays a smaller role in CASs than in reductionist worlds, since the whole need for prediction is less important. With CASs we try to move from the idea of prediction to an analogue concept involving exploration and understanding. As Robert Oppenheimer noted, 'You don't understand QD, you just get used to it'.

From this perspective, we can see that ballast water is to our coastal ecosystems, as the ice ages were to terrestrial ecosystems. We have created connections between ecosystems that were previously unconnected. Thus we are not dealing with incursions or invasions, with their connotations of activity at the periphery, at the edge of the system. We are really dealing with a deep systemic change in that the fundamental connectivity of the system has been altered.

History tells us that such connections produce surprise and novelty. For example, the mammalian fauna of North America, the product of repeated connections and disconnections between it and Asia as well as South America, is neither predictable from any biogeographic theory nor the result of some random mixing process. It is first and foremost understandable only after the fact and through the use of historical rather than scientific tools.

Provenance and prediction

Seeking provenance or prediction, reaching back into the past or forward into the future, are probably the two hardest things we call on science to do. Typically, reductionist science seeks both the past and future in the present, by attempting to filter out the singular nature of time, leaning towards some platonic ideal, where time is either absent or else cyclical. Complex systems theory accepts the historicity of the world and reaches back with historical tools and forward with exploratory ones.

In the world of incursions and ballast water, reductionist approaches are reduced to simplistic 'risk analysis' approaches and a narrow view of individual events. Complex adaptive systems approaches take a broader view of the system, and acknowledge a hierarchy of embedded historical singularities, from the establishment of new system connections to individual incursion events. This allows for the exploration of the future and the development of adaptive strategies.

References