Can the future be known? I

Introduction

Well, there is good news and bad news. The answer is both yes and no. Simultaneously. Simultaneously in the same sense that Schrödinger's cat is at once both alive and dead[1].

With the strictures of time, I will focus on why the answer is no, letting the case for yes emerge, and I shall have more to say about the idea of emergence later.

But first I want to turn to how this can be. I will leave you to decide whether Schrödinger's poor cat is merely a metaphor - a whipping cat, perhaps - or something deeper.

The question is coloured by 'future', but pivots on 'known'. Let us refine the question first by getting some understanding of colour, before we deal with the rhythm and movement.

We often depart for the future from the past, that is we consider the past, present and future to lie along some sort of metaphorical line in a geometry of the world which includes both space and time as axes - a habit of thinking since at least Descartes in the 17th century, and probably Euclid in 300 BC. In this geometry we say with L P Hartley[2], the author of that minor classic, The Go-Between: 'The past is a foreign country: they do things differently there', and then, repeating this mantra for its pigeon pair, the future, get on with the difficult technical business of hindcasting or forecasting.

On such differences, I take a different view, closer to Scott Fitzgerald[3] than Hartley - 'Let me tell you about the very rich. They are different from you and me.' I want to argue that the thing that makes the future different from the past, and, indeed, the present, is its richness, its potential - that while it is may be a foreign country, it is a peculiarly different one, and one not reachable with the geometries or maps of Descartes or Euclid.

And what of our pivot, that subtle verb 'to know'. I take science to be the state of knowing, so, for me, to know something means to understand, in some sense, that something to be a true fact about the world. Thus I am not talking about knowing that I must go to the dentist tomorrow, because that is something I can forget. I am talking rather about knowing, say, the Earth is part of the solar system. Even if I forget that particular contingent fact of the universe, it will still be known by others, since it is, in some sense, the assimilated knowledge of the culture.

The trouble is that science has been so spectacularly successful with this way of knowing some futures that it is tempting to think it can be used for most or even all futures. To understand the problem that this creates we need to be very clear about the sorts of futures I am talking about, the sorts of futures that science is good at, and exactly how it got to be so good.

Science has been terrific at knowing the future of those systems where past, present and future do, as a first approximation, lie along a line in some Cartesian geometry. Indeed since at least the time of Ptolemy, in the third century BC, we have known how to predict astronomical phenomena such as eclipses using just such geometries. Astronomical systems are the canonical examples of highly predictable systems which broadened in the succeeding millennia to embrace other dynamical physical systems, such as gases, machines and electricity. We can reach into the future with both precision and accuracy in such systems, given some knowledge about the state of the system in the present.

Science got to be good at predicting the future of such systems by using models. That is, science invented a fantastic trick: by creating a simpler caricature of the real system - one stripped of those bits extraneous to the matter at hand - in its own caricatured time and space, it was able to 'run the model' into the model future (or past) and predict what would happen in the real world. Notice that I have not mentioned the M-word yet. Models do not have to be mathematical, although they often are: they can be mechanical, electrical, geometrical, pictorial or even narrative as well. All they have to do to qualify as models is to have the character of a caricature - being stripped down to the essential elements - and having their own world. Science found some models to be so powerful they were called physical laws.

Those systems whose future science successfully predicted with such models have in common the fact that they are all rather simple. They are made up of relatively few distinct entities - planets are all much the same when it comes to the way they orbit the sun - and their interactions tend to be limited - Newton's inverse square law of gravitational attraction ensures that, for all practical purposes, a planet is only affected by the very near or the very large, all the rest may be ignored.

But these are by no means all the interesting systems in the universe. They may, in fact, be the least interesting when compared to complex systems, and in particular, to complex adaptive systems. Complex systems are characterised by, well, complexity. Where simple systems have few entities, complex systems have many; where simple systems have few interactions, complex systems have many. The weather, the oceans, and the earth beneath our feet are all examples of complex systems.

Complex adaptive systems, such as all living things and their parts - cells, say, or immune systems - and their assemblages - societies, economies, ecosystems and so on - take this complexity further in exhibiting the peculiar characteristic of learning, of evolving, of adapting. We speak more easily of the behaviour of such systems to denote this flexibility, where we speak naturally of the dynamics of simple systems to denote their relative fixedness.

Since we are interested here in those systems where man is coercive or at least involved, we are naturally interested in complex adaptive systems.

So our question, Can the future be known?, becomes Can we reproduce science's predictive triumphs with models of complex adaptive systems?

A brief history of time

Like Blaise Pascal[4], I do not have the time to make my consideration of time any shorter than Hawking, so, commending him[5] to you, I will sketch my central point. And that is: for complex adaptive systems at least, the past, present and future do not sit side by side along some geometer's line like crows on a barbed wire fence. Decartes' geometry does not work for these systems because the future is not to the present as the present is to the past.

There are many futures, but there is only one present and one past. In a sense, many futures exist simultaneously, but only one becomes the present. When the present occurs, these many futures collapse into one, the one which becomes the present.

Or at least, they behave as if they do so exist. I am describing the future of complex adaptive systems, of course, as if it were like the future of quantum mechanical systems, where many, but, importantly, not infinitely many, possible quantum states exist simultaneously - and I use exist in a deliberate, real sense - in superposition, and which then collapse to the one state - the present - on observation.

It is not a key issue for this discussion whether I am describing a metaphor or something deeper. What I am describing is the bones of an alternative model for the way complex adaptive systems behave with time. I am describing a time very like the time of Jorge Luis Borges Garden of Forking Paths[6].

An experiment

Allow me, as a scientist, to do an experiment to demonstrate what I mean. Let me clap my hands.

We have moved instantaneously from the present to the future leaving behind, forever, a past. The future, in that long ago before I clapped my hands, is now our present, as that present is now added to our past.

The important thing is: What changed? The answer is: Not much. Perhaps it is even: Nothing - with the caveat:At least to a first approximation. But let us settle on not much.

For all practical purposes (which is the ordinary way of saying the much more grand and scientific to a first approximation), this future we discovered in our experiment is pretty much like the present we knew those few moments ago, and that present looked pretty much like the recent past.

Let's be good scientists and repeat the experiment. I'll clap my hands again. Scientists love repeatable experiments!

Within measurement error (which is another high-falutin way of saying for all practical purposes), we just got the same results. This is good, because we now have a trend. Scientists love trends even more than repeatable experiments! The great turn-of-the-century mathematician, Henri PoincarČ, said that with two points he could fit a straight line, with three a curve, and, if given five, he could build you an elephant.

With such a trend, we can extrapolate the future from the present, because we just discovered that the future is like the present - just as I can draw teeny weeny little straight line segments on this whiteboard. We can even be sophisticated: we do not have to settle for straight line extrapolation, but can draw curves. These extrapolations will be familiar to anyone who has ever read a company financial report, an economics paper, or a climate prediction.

I call such extrapolations Heil-Hitlers, they just shoot upwards. And Heil-Hitler extrapolations for complex adaptive systems are always wrong. Wrong whether straight or curved - because straight lines are just special cases of what mathematicians call curvi-linear systems; and wrong whether two or three (or more) dimensional - because the dimensionality doesn't change the argument one whit, even if higher dimensional pictures do look terribly clever.

Why we are always wrong

We all fell for an old trick when we interpreted our experiment. We misused one of the oldest tricks in the bag of mathematics and logic: proof by induction. Consider this experiment: I start with the integer 1, and add 1 to it to get 2. I then add 1 to that result and get 3. I find that if I continue this I can reach successive integers, so I conclude, by induction, that I can reach all the integers by this process of successive addition.

By such a process of induction, the Greeks invented arithmetic and geometry, and Newton, the infinitesimal calculus. This idea of proof by induction is one of the most powerful weapons in the armoury of mathematics, but like all weapons, is dangerous if misused. When we apply it outside mathematics, that is, when we apply mathematics to the real world, we are saying that mathematics is, in some sense, a model of the real world. We are saying, in this case, that the world really works in such little inductive steps. Our problem is that while mathematics may a model of the real world, the real world may not be a model of mathematics - the relationship between the worlds of reality and mathematics may not be transitive.

Once more, we have arrived at a deep question, but once more we can skim over it, taking the relationship as either metaphor or something deeper as we like, without undoing the argument that induction, of the strictly reductionist variety just described, may work with simple systems, but will never work with complex adaptive systems.

Indeed, this is so important that we will return later to what it is that replaces induction and deduction, those two towering pillars of reductionist science, in the science of complex adaptive systems.

This is getting far too abstract. Time for an example. Think of yourself. You are as good an example as any of a complex adaptive system. If induction worked for you, could you run the tape backwards, as it were, in teeny, weeny little inductive steps, and see successive frames of your life, back until you were an egg? Of course not! Sure, things do change in your life in little steps, but there are also some sudden big changes, some more subtle qualitative shifts, some accidents. As Ned Kelly[7] said on the Pentridge scaffold: Such is life. The best an inductively based model could do would be to wind the tape back until you looked like a teeny, weeny little person the size of an egg - in fact in the seventeenth century, at the height of such nonsense, scientists even thought they could see just such a little homunculus inside every sperm.

What if we started with a sperm, the one that joined the egg to create you, and ran our inductive process forwards? Would we get you? The best we would get would be a rather disgusting gigantic eyeless tadpole, the less said of which the better.

So we have got to a peculiar position intellectually. We believe, on the one hand that we get to the future in a series of small increments such that in the next instant, we are much as we were in the previous one. Experience tells us so, and induction reifies it. We also believe that, on reflection, we didn't get to here by any sort of smooth incremental process, but that our life's path had jumps, bifurcations and so on. In short, we believe two incompatible things at the same time.

Do not be alarmed, we are not alone! The great philosopher, Willard Quine[8], has also noticed this:

'To believe something is to believe that it is true; therefore a reasonable person believes each of his beliefs to be true; yet experience has taught him to expect that some of his beliefs, he knows not which, will turn out to be false. A reasonable person believes, in short, that each of his beliefs is true and that some of them are false. I, for one, has expected better of reasonable persons.'

Why should this be so?

We are in this mess because we live in what we might call a middle number world. By this I mean we don't live at either extremes of the scales of space and time of the universe, but rather in the middle. Our world is built neither in length scales of Ångströms[9] or light years[10], nor time scales of nanoseconds[11] or billions of years. This causes problems by itself because at these extremes, prediction is relatively easy. In the very short run, things generally only change a little, so induction actually works. In the long run, as Maynard Keynes pointed out, we are all dead, which is the economist's way of saying that the ultimate entropic fate of the universe is known with some certainty.

But more importantly than this, we live in a world with a middling number of entities, neither one or twos nor uncountable millions, but rather hundreds and thousands. In this world of complex adaptive systems, we need to know the names and natures of these middling numbers of entities, because their differences count - the hero counts in history. We do not have the luxury of needing only to know a few of these entities well - as in celestial mechanics. Nor may we hide behind the assumption that these assemblages can be treated with the laws of probability as statistical ensembles of nameless, unknown, identical individuals, that is, as individuals shorn completely of their individuality - as in the dynamics of gases.

Our world, the world of complex adaptive systems, hovers tantalisingly between these two extremes. This is what some commentators trendily call the edge of chaos.

These middling numbers of entities lead to a curious result: to an uncountable number of interactions. To understand this, we need to understand that there is a boundary to counting in the real world, unlike in the ideal world of mathematics where, as we saw earlier, induction will allow us to reach an infinity of numbers. As explained by Brian Rotman[12], because counting has a cost in the real world - the batteries in my calculator might run down after a while, the ink might run out in my pen - I will eventually reach a point where what I am doing has changed so much that it no longer constitutes counting. This boundary doesn't go away just because we use a computer, or even if we use the largest conceivable computer, one that could consume all the energy of the universe, for example. The boundary just pushes out a bit, but it is still there. It turns out that this number, for a computer that consumes all the energy of the universe, is about 10⁹⁶.

Now this is not a very large number when you consider it. It[13] is about 68! Now the numbers of ways a bunch of entities can interact is roughly equal to its factorial. So Rotman's result says that we have no hope, no possibility, of ever enumerating all the ways even 68 different entities may interact.

More generally, we may say that with complex adaptive systems, the future state of the system is not in principle predictable because it is not in practice computable.

There is no way out of this dilemma, but many scientists would have you believe that there is. These Panglossians recall the old non-PC Air Force slogan: 'What this Air Force needs is more beer, longer runways and bigger women'. Scientists from geological, economic or biological survey agencies will all claim more and better data will solve the problem, while scientists from analytical agencies will claim that bigger models, bigger computers, more parameters, and above all more fudge factors will save the day. And of course, weathermen will have two bob each way. Sadly, neither more data or bigger models will help us cut this Gordian knot.

And it is so easily demonstrated! Just ask one of these optimists to run their models backwards to capture the past.

These arcane musings have some quite practical and important consequences, as I will now show.

What is to be done?

The words, of course, are Lenin's[14] and they neatly define what we mean by strategy. The idea of strategy changes when we shift focus from simple systems to complex adaptive systems, because the ideas of induction, deduction and prediction - all of which bear directly on strategy - also change.

One of the most pervasive[15] linear assumptions (really, underpinning much of reductionist science so thoroughly that it is not even noticed) is that induction, the generation of general principles, is equivalent to the generalisation from observations, while deduction is its obverse, and equivalent to the creation of predictions of instances from general theory. CAS theory suggests otherwise and that induction/deduction are themselves neither simple nor a pigeon pair.

When CASs are involved (really always, since we are always involved), induction is an emergent process where ideas emerge, some living and some dying, from an interaction between all the CASs. This is much closer to the Einsteinian spirit of pragmatism of the creative act, and its need to solve problems.

'Physics constitutes a logical system of thought which is in a state of evolution, whose basis cannot be distilled, as it were, from experience by an inductive method, but can only be arrived at by free invention. The justification (truth to content) of the system rests on the verification of the derived propositions by sense experiences, whereby the relations of the latter to the former can only be comprehended intuitively.'

Unlike Newton, Einstein was not a mathematician of the first rank, but rather a first rate scientist. This quote shows he was more concerned with our world, the real world, than the world of mathematics.

Again deduction plays a smaller role in CASs than in reductionist worlds, since the whole need for prediction is less important. With CASs we try to move from the idea of prediction to an analogue concept involving exploration and understanding. As Robert Oppenheimer noted, 'You don't understand QD, you just get used to it'.

What then does a model of a complex adaptive system produce if it doesn't produce predictions?

The best models just live their lives, display 'interesting' and sometimes surprising behaviours and interact with the modeller, each learning from and adapting to the other. It is that learning and adaptation that the modeller can then take away to help him understand the real world.

Thus the strategy we might induce from models of simple systems is 'Don't put all your eggs in the one basket'. This simple nostrum underpins all insurance, portfolio management and most so-called rational approaches to the unknown.

But CAS theory tells us to look at that great teacher, evolution. After all the Burgess Shale contains as many extinct r strategists (the generalists) as K strategists (the specialists) and teaches us the lesson that no single strategy can protect you from the future. Extinction awaits all systems, the present survivors are those that adapted and changed.

Thus in a model of a complex adaptive system, we would hope to see the emergence of a risk coping strategy - one that we ourselves might learn from - where we would see the induction of a risk amelioration strategy from a model of a simple system.

There is no magic here, no anti-scientific stuff. We can explain most of the behaviours that we see in complex adaptive systems in normal scientific terms. No fundamental physical law is violated. The only difference is that we can only do this explaining after the event, we cannot use this explanation to help us predict the future.

While explanation may lead to prediction in simple systems, they are differently related in complex adaptive systems. And on that rich difference I return to Scott Fitzgerald and end.

References