It's the state of the network, stupid!

Back to the discussion on Artificial Intelligence and Neural Networks. This blog hosted some meanderings and thoughts on AI and how this relates to NN. I read books from Steven Pinker and others regarding NN's and I understand how these work from a helicopter view (I even implemented some). I then analyzed that the perspective of NN's on AI is probably horribly wrong, if you compare biological networks against mechanical ones.

Computers were from the very start designed to work on input and then produce output. Output is a very binary-like state. It's either this or that, it's a range of numbers and the output is generally an exact number. There's not really a way to represent an answer that represents two (slightly) different states in one answer by itself.

This morning, I woke up and considered that this general approach that was taken as part of AI way back is probably wrong. Even though computers are supposed to produce output that can only be interpreted in one single way, the "output" of human brains doesn't really exist as output per se. I'm more thinking of answer and thought as a kind of "state of the network" at some point in time. The frequency of thought is given by the "frequency" of the network, although this seems a very weird term to use for biological networks. It's probably totally independent.

If you look at CAT-scans though, you'll see something interesting. Not all neurons are active at all points in time (very contrary to mechanical networks, which generally have all their nodes and parts connected, so that turning one part will turn and impact another). And the granularity of our analysis on the human brain is not at neuron level, but at the level where we see a general number of neurons receiving activity. So if one neuron A next to neuron B is fired, only A would be active, but B would not be assessed.

So it's like regions of interconnected neurons are active at one sweep, not the entire network. And there's no output like a machine, only a list of active and recently active neurons. Every sweep, the active list is modified and moved back into a ring of memory.

If we reconsider neurons as nodes in a network and replace them with symbols instead, we can probably come close to a logical representation of a thought network. So, a neuron by itself doesn't represent anything, but impacts something physically. A symbol is highly representative of something, but doesn't necessarily impact anything, it is only connected to other symbols.

The symbolic network is then like a neural network, only it works with nouns, with symbols. The symbolic network allows an infinite number of nodes to be added, as long as there exists a process to interconnect symbols as soon as there is a relation to be determined between them.

Now, imagine what happens and assume this symbolic network is properly connected. When you mention a word car, or the smell of exhaust gas, or a picture of a car, those symbols are activated. The joint activation of car, exhaust gas and the picture should activate a symbol of car (without annotation) as a concept, so that the network understands that car is being discussed.

If you now introduce a text with nouns and verbs and assuming the computer has grammatical analysis capabilities, you can process this text within the symbolic network and at the end of some paragraph / passage of text, the network has a certain state of activity. Some regions are highlighted and other regions are black. If you'd keep a list of symbols that are activated, then you could store that list (region) as a representation of the text.

So, the objective is not to store the text word for word, but to store the associations and context of the paragraph. Mentioning the words in a search term would probably produce the text again and the more aligned with the paragraph it is, the more likely it is to be found.

Memory is also important. There are actually different rings of memory in this model (short-term and long-term are too generic). Reading a passage would store the gist of that passage into a different symbol. The gist are basically those nodes that had the highest activation of a certain paragraph after the cycle is completed. So storing the gist of one paragraph with another may develop a description of a document that is highly descriptive. It's not necessarily the word that is mentioned, it's the concept and the relation to other symbols. It's possible that a symbol is highly activated that was not explicitly mentioned in the text.

The symbolic network is the representation of nouns of our language, but verbs are the activities, the manipulations in our mind of those symbols. It seems then that within the context of this blog post, the verbs correspond to real intelligence (which is not described in this post yet). The nouns are just perceptions and mapping them to symbols. Real thought, the one that creates (is creative) and can reproduce and come to certain conclusions is a totally different matter. That sounds like real artificial intelligence.