In the last post there was plenty about Gödel but not much about Grammar and Go-the-game. Time to pay my debt!

Basically I said that Gödel’s results proved that no fixed set of facts and rules can on their own form the basis of mathematical knowledge. I said that hard-earned experience is indispensable. That mathematics is ultimately an experimental science. (This is not the usual take on Gödel’s work.)

But grammar? For natural languages, it’s the same story. Forget about semantics (meaning). Just the syntax of a natural language like English is infinitely rich and can’t be described by any manageable set of facts and rules. The same goes (sorry) for Go-the-game-not-the-programming-language. To master them you need judgement and experience.

Does this mean facts and rules are not as important as we might think? Actually no, they’re indispensable. In fact they are a vital part of what makes us human!

Formal grammars were invented by Chomsky and, independently, the Algol committee, to specify Algol and describe natural languages. They worked spectacularly well for Algol and got off to a good start for natural languages.

For example, one rule that covers a lot of sentences in English (and other languages) is

<sentence> ::= <noun phrase> <verb phrase> <noun phrase>

But already you have trouble because in many languages the verb phrase has to agree with the noun phrase in terms of number. So you need two rules

<sentence> ::= <singular noun phrase> <singular verb phrase> <noun phrase>
<sentence> ::= <plural noun phrase> <plural verb phrase> <noun phrase>

In Russian (in the past tense) the verb phrase has to agree with the noun phrase in terms of gender (there are three in Russian). Six rules.

Let’s stick to English and concentrate on noun phrases. One big thing to deal with is the definite article “the”. Native speakers don’t think about it, but there are rules for “the”. For example, it does not precede personal names, like “John” or “Alison”. Or names of organizations, like “IBM”. Oh wait, what about “The Government” and “the BBC”? Hold on,  you don’t say “the NBC” … ???

I have no idea what the rules are. I’ve had many students whose native language (Chinese, Farsi, Korean, … ) has no definite article. I often have to correct their usage and it seems they are always coming up with new ways to get it wrong.

So there seems to be a kind of incompleteness phenomenon here. No matter  how many facts and rules you discover, there’s always a sentence that is idiomatic but not covered by these facts and rules.

This is what torpedoed early AI efforts in natural language processing. It was based on grammar and logic and failed because you never had enough facts and rules.

The first efforts at playing games like Chess or Go  were also based on facts and logic. The main rule is that the value of a position for one player is the negative of the value of the least favourable (for the other player) position arrived at in one move (whew!).

That rule, and a whole bunch of facts about who wins a terminal position, in principle is enough. But not in practice.

So instead you need heuristic rules to evaluate positions. (In Chess, having passed pawns, controlling the centre, material superiority etc etc). IBM managed to make this work for Chess but for Go it was hopeless. Too many possible moves, too much context to take into account.

And yet there is AlphaGo, which has beaten the world champion. How does it work?

I don’t know. It uses neural networks to process hundreds of thousands of professional games and millions of games it plays with itself. The only facts and rules that humans give it are (as I understand it) the rules of the game. Maybe not even that – the explanations of AlphaGo are vague, probably because of commercial secrecy.

However, I think I can explain the success of AlphaGo (and, recently, Google translate) by an appeal to human psychology. Specifically, to the notions of conscious and unconscious.

It’s generally agreed that the brain works in both conscious and unconscious modes. Most of the processing is in the unconscious mode and we are (needless to say) unaware of it. How does the unconscious work? Not clear, though it may involve thrashing out contradictory tendencies.

The unconscious communicates with us through feelings, intuitions, hunches, judgement, perception, aesthetics, reflexes …

The marked stone leaves a bad taste

Anyone who has taken Go seriously will be amazed at how experts talk about the game. They use concepts like strength, thickness, good and bad shape, even (I’m not making this up) taste. Teachers encourage their students to play quickly, relying on instincts (reflexes). Learning Go is not so much about memorizing facts and rules as training your unconscious. Maybe AlphaGo works the same way, by simulating an unconscious and training it.

What then is left for the conscious? Guess what – facts and rules.

I’m convinced that the conscious, rational part of the mind works in a machine-like fashion, using and manipulating facts and rules, devising and following step-by-step protocols (algorithms).

I’m not saying that people are machines. We rely on our unconscious, which apparently does not work sequentially. The conscious and unconscious work together and make a great team. Only if you consciously ignore your feelings do you become a soulless robot (though there’s a lot of that about).

For example, in mathematics we first discover facts and rules by insight based on experience. Once we’ve found some we have confidence in, we then consciously apply them, draw consequences through step-by-step reasoning and leap way ahead of what we could discover by experience alone.

It’s this teamwork that give us such an advantage over animals, who act almost completely unconsciously. It’s what makes us human. It gives us the freedom to choose between doing what we feel like doing – or, if it’s not the same thing, doing what is best. It gives us free will.