Meaning and information

I don't remember meeting him, but, when I was a child, we often used to drive past a house belonging to my father's old colleague, 'Sticky' Glew. (My mother had been introduced to him, and had trouble deciding how to address him. She wisely opted for 'Mr Glew'.) It amused my father that his friend had written a dissertation on 'the meaning of meaning'.

"For thousands of years," writes Seth Lloyd (in his book, Programming the universe (Vintage, 2007)), "philosophers have tried to determine what 'meaning' means, with mixed success." (p. 24)

Lloyd's work (in quantum computing and quantum communication systems) is based around the technical concept of information. But strings of bits and so on mean nothing in themselves: they are meaningful only if they can be interpreted. "Meaning is defined only relative to a scheme of interpretation ..." (p. 25)

"Consider the string of bits ... : 1001001 1101110 0100000 1110100 1101000 1100101 0100000 1100010 1100101 1100111 1101001 1101110 1101110 1101001 1101110 1100111. Interpreted as a message encoded in ASCII, this string means 'In the beginning'. But taken on its own, with no specification of how it is to be interpreted, it means nothing other than itself." (p. 25) And, of course, 'In the beginning ...' is interpreted according to the conventions of the English language. As Lloyd points out, natural languages are rich in ambiguity, which is "a key aspect of poetry, fiction, flirting, and plain everyday conversation." (p. 27)

But sometimes, in order to understand basic concepts - like meaning - it is useful to strip away complexities and ambiguities and look at simple models, as Wittgenstein did in his account of language games. Imagine a simple language game in which a builder says "Block" and the assistant hands him a block, or "Slab" and the assistant hands him a slab. The meaning of each expression is to be found in the action the expression provokes.

Lloyd relates Wittgenstein's idea to computers and computing. "The meaning of a computer program written in a particular computer language," he writes, "is to be found in the actions the computer performs as it interprets that program. All the computer is doing is performing sequences of elementary logic operations, such as AND, NOT and COPY ... The computer program unambiguously instructs the computer to perform a particular sequence of those operations. The 'meaning' of a computer program is thus universal, in the sense that two computers following the same set of instructions will perform the same set of information-processing operations and obtain the same result." (p. 26-27)

Meaning, then, is interpreted information. We don't need a theory of meaning such as philosophers have attempted to build. Philosophy (most notably the philosophy of language and metaphysics) has drifted into unproductive areas reminiscent of scholasticism in which intellectual work is done without first ensuring that there is an important intellectual task to address.

Of course, the word 'meaning', like many English words, can be used in different ways, and a careful analysis of the context of use will reveal subtle - and not so subtle - differences. One sense of the word - rather different from most of the others is 'general significance' or 'point' or 'purpose', as in the sentence, 'My life seems to lack meaning.' I have not been talking here of this generalized kind of meaning, but of the ordinary - and primary - uses of the word in relation to information and communication. And, understood as interpreted information, it seems to me a perfectly clear and unmysterious concept. Trying - as some philosophers do - to make a big issue of what they call 'aboutness' or 'intentionality' (which doesn't mean what non-philosophers think it means) constitutes unnecessary mystification.*

The remarkable advances in computing and information and communication technologies during the last 70 years have thrown up many real problems of a fundamental nature, but they require scientific knowledge (not just knowledge of formal logic and philosophy) to address. In particular, the parallels between thermodynamics and information theory are clearly rich in new problems which would benefit from informed reflection. For example, all matter and energy is subject to the laws of thermodynamics, but all matter and energy - everything there is - is also subject to the laws of information. Information theory appears to be a more fundamental and all-encompassing theory than thermodynamics (which can now be seen as just a special case of information theory).

Though I doubt the value of much recent philosophical work in the areas of language and meaning, some very important foundational work has certainly been done by philosophers, philosophically-inclined mathematicians and logicians. The key advances were made during the 19th and early 20th centuries. The work of George Boole, Giuseppe Peano, David Hilbert, Bertrand Russell, A.N. Whitehead and many others provided the conceptual tools which made the development of the electronic computer and sophisticated communication technologies possible. Claude Shannon's development of what he called a 'mathematical theory of communication' and which has come to be known as information theory was based largely on the work of George Boole.

Information has come to be seen as physical - it is no longer seen as abstract, disembodied and unquantifiable. It is always tied to a physical representation: a mark on paper, a charge, a spin, a sequence of bases in DNA. Information processing occurs not just in computers and brains but throughout the physical world.

In 1961 Rolf Landauer came up with a principle with (it is said) startling implications: that one essential information processing operation (erasure) cannot occur without causing heat to dissipate into the environment (thus increasing the entropy of the universe). The processing of information is a thermodynamic process (just as thermodynamic processes are informational) and erasure is an irreversible operation. Negation can be reversed by a second negation. Addition can be reversed by subtraction. But erasure cannot be undone.

I will resist the temptation to discuss the implications of Landauer's principle (which I believe are not good!). I am a layman in these matters, but the role that information processing seems to play in every aspect of nature intrigues me. I am trying at least to understand the fundamental principles, and to relate the sort of thing one learns within the context of philosophical logic - e.g. that there are any number of alternative systems of logic - to the apparently more constrained context of real-world information processing.


* Much philosophical work in recent decades in areas such as ethics, metaphysics, the philosophy of language and even logic has been done by people with a commitment to a religious view of the world (or at least with an anti-physicalist orientation), and has been motivated (I believe) by an attempt to undermine physicalism and to save a space for the spiritual (broadly interpreted). Such a motivation does not invalidate the work, but I personally don't think a convincing case against physicalism has been made.

Through a crystal darkly

In previous remarks on randomness and computation, I mentioned the work of Gregory Chaitin, a mathematician and theorist who has written and spoken (he is a brilliant speaker) extensively for both specialist and general audiences. Chaitin's technical work is highly regarded, but his interpretations and extrapolations are sometimes a little idiosyncratic and he is inclined to sound a bit New Agey at times. (He is rumored to receive help in his thinking from a giant crystal!)

Paul Davies (a physicist and writer) is, by contrast, sober and restrained - even a little pedestrian by comparison - but he is a reliable guide within his areas of expertise. I recently came across a foreword by Davies to a book of Chaitin's essays* in which Davies gives his perspective on the significance of Chaitin's work and its implications for physics and our view of the world generally.

Chaitin (who had been obsessed from his childhood years with Kurt Gödel's incompleteness theorem) "greatly extended the scope of Gödel's basic insight," writes Davies, "and recast the notion of incompleteness in a way that brings it much closer to the real world of computers and physical processes. A key step in his work is the recognition of a basic link between mathematical undecidability and randomness. Something is random if it has no pattern, no abbreviated description, in which case there is no algorithm shorter than the thing itself which captures its content. And a random fact is true for no reason at all; it is true 'by accident' so to speak ... Chaitin was able to demonstrate that mathematics is shot-through with randomness ... Mathematics, supposedly the epitome of logical orderliness is exposed as harboring irreducible arbitrariness." (p. vi)

"[M]athematics contains randomness - or accidental, reasonless truths," Davies explains, "because a ... universal Turing machine [an idealized computer], may or may not halt in executing its program, and there is no systematic way to know in advance if a function is computable (i.e. the Turing machine will halt) or not." (p. viii)

But this limitation on what we can know or predict (known as Turing uncomputability) applies not just to mathematics and computers but also to scientific theories. On Chaitin's view, a scientific theory is like a computer program that predicts our observations (the experimental data).

Indeed, in the words of Paul Davies, " ... we may regard nature as an information processing system, and a law of physics as an algorithm that maps the input data (initial conditions) into output data (final state). Thus in some sense the universe is a gigantic computer, with the laws playing the role of universal software." (p. viii)

And if the laws of physics are computer algorithms, there will be randomness in the laws of physics stemming from Turing uncomputability. But, according to Davies, the randomness will, in reality, be "even more pronounced than that which flows from Turing uncomputability." (p. viii)

He points out that the real universe differs in a crucial respect from the concept of a Turing machine. "The latter is supposed to have infinite time at its disposal: there is no upper bound on the number of steps it may perform to execute its program. The only relevant issue is whether the program eventually halts or not, however long it takes. The machine is also permitted unlimited memory ... If these limitless resources are replaced by finite resources, however, an additional, fundamental, source of unknowability emerges. So if, following Chaitin, we treat the laws of physics as software running on the resource-limited hardware known as the observable universe, then these laws will embed a form of randomness, or uncertainty, or ambiguity, or fuzziness - call it what you will - arising from the finite informational processing capacity of the cosmos." (pp. viii-ix)

There are, it seems, different forms or levels or randomness. The 'mild' form which - as chaos theory shows - is implicit even in classical, deterministic physics; the pseudo-randomness which can be generated by simple computer algorithms; the well-known randomness inherent in quantum mechanics; and perhaps the deepest levels of all stemming from proven features of idealized computers (Turing machines) and from seeing the universe itself as a giant computer - one with specific limitations on its processing capacities.

These are difficult (and to some extent speculative) ideas. But I think they are worth pursuing and may even have profound implications for how we see ourselves and our world.

It is, of course, impossible to draw definitive political or metaphysical conclusions from them, but, if the ideas are sound, there will be such conclusions to draw.

Let me just mention two thoughts which come immediately to mind: Chaitin's and Davies' notions are utterly incompatible with any political ideology which attempts to predict, plan and control human affairs; and they also appear to undermine perspectives which incorporate notions of a providential force operating behind the scenes and impinging on natural processes, historical events and/or individual destinies. 


* Thinking about Gödel and Turing: essays on complexity, 1970-2007 (World Scientific, 2007).

The randomness at the heart of reality

Is reality ultimately based on randomness? How one answers this question ultimately colors one's outlook (I suggest) in deep and subtle ways. Of course, there are many ways one could approach the issue and there are ambiguities in the question itself. But I am drawn to such questions as this (as a moth to a flame?) and, since I read and think about them, I might as well write about them here from time to time.

Here, then, are a few notes about Vlatko Vedral's view of the issue ...

Vedral draws a distinction between "classical superficial randomness" (e.g. coin tosses) and "quantum fundamental randomness" (see Decoding reality (OUP, 2010), p.163). Randomness approximates to unpredictability and much in our world appears random because it is impossible to predict in practice even if in theory one could do so using the methods of classical physics (if one had all the relevant data etc.). But the quantum world is different. No prediction can be made (even in theory) of certain quantum events. Quantum theory embraces randomness, and sees some quantum phenomena as random in a fundamental sense.*

We can think of scientific theories, Vedral writes (p. 166), as computer programs "with the output being the result of whatever experiment we are trying to model. We say that our theory is powerful, if we can compress all sorts of observations into very few equations."

But any theory will be finite and will (as Gregory Chaitin first fully realized within information theory) only produce a finite set of results. "In other words, there will be many experimental outcomes that could not be compressed within the theory. And this effectively implies that they are random." (p. 167)

Is the randomness in quantum theory due to the theory's incompleteness - our lack of knowledge of a more detailed deterministic underlying theory - as some people think? Or is "randomness inherent in the Universe, and therefore ... [an essential] part of any physical description of reality? Randomness could simply be there because our description of reality is always .... finite and anything requiring more information than that would appear to be random (since our description could not predict it)." (p. 167-168)

Vedral states what he sees as a "very profound conclusion" that this view implies: "that randomness in quantum physics is far from unexpected - in fact according to this logic it is actually essential. Furthermore, it would mean that whatever theory - if any - superseded quantum physics, it would still have to contain some random features." (p. 168)

To me it matters (or seems to matter) whether or not randomness is at the heart of things. Does it matter to you?

* I have revised this passage slightly in response to a comment.