P=NP?

A new post on Silvio Meira's blog today. Apparently, one of the math professors over at Pernambuco in Brazil is going to post the proof that P=NP.

http://silviomeira.blog.terra.com.br/p_np_sera

Can't wait to see the result. It would have very large consequences for everything and computers will definitely be much, much, much more helpful than they are today. Stay tuned for that story.

I'm following it as well and see what happens!

Three Levels of Analysis

One interesting book of Pinker on Cognitive Science discusses the three levels of analysis, it is also on Wikipedia (since it belongs to the field of CS). Anyway, these three levels represent three layers in the OSI model of computing. The physical layer, the algorithmic layer and the computational layer. The algorithmic layer would be compared to the Operating System and the computational layer to the behaviour (which would be applications making use of the infrastructure).

Of course, when mapping these levels to biological counterparts, they map to neurons and so on.

The interesting statement is that it is not only required that you understand the workings of one layer individually (to be able to make sense of something), but you'd need to understand the interaction between the levels of analysis in order to work out how something works.

There are some potentially large pitfalls there. We make assumptions that a neuron has a certain use and can never have more than one use at the same time. I have no information at this time to make any further statements on that though.

One of the questions in the books is what intelligence really is, and it is then purported as some kind of computational machine. Computation is often associated with algorithms, formula and maths.

I feel happy about people saying that mathematics and algorithms somehow describe the world (or can be good means to communicate difficult issues or describe some kind of event), but I don't think it's a good idea to turn this upside down and then state that the world is mathematical in the first place. It's way more dynamic then that. It's maths that describe the world (and then somewhat poorly, as some kind of approximation).

Although very complex things can be described with a sequence and combination of many algorithms together, this presents enormous problems. First of all is that it makes the world deterministic (although incredibly complex with algorithms connected and influencing together) and second is that in order to come to a full implementation, you'll need to understand absolutely everything and model it in some kind of algorithm. That sounds pretty much impossible and a 'dead path'. I think AI needs something more dynamic than that.

There was a lot of enthusiasm on Neural Networks. I've used them too and I like how they operate, but in the end a network with its factors is only useful for one single task. Another task cannot be handled by it, unless it is retrained. So those are very limited as well, plus that I find the speed and method of the human brain for learning immense. Another limitation of NN that I find is that they require inputs and outputs and have a fixed composition for the problem at hand (a number of layers and a fixed number of neurons and synapses inbetween).

So, NN's are also deterministic and limited to its purpose. What and how should something be designed so that it can seem to learn from something and also start reasoning with its knowledge? Reasoning in this context to be interpreted as a very wide term. I explicitly did not use the term compute to make a difference. A Design is often very limited, limited to its area of effectiveness and to the knowledge at hand at the time. The design is limited to our disability to design indetermine things, items that may produce inconsistent and unexpected results.

When we design something, we do that with a purpose in mind. If this is the input, then the result should be that. But this also limits the design to not be capable of doing anything more or different from what it was designed to do.

How difficult is it then, with our minds focused on algorithmic constructs (consistent results), if we are just now trying to work out a design for something that may produce inconsistent and indeterminate results? It's the toughest job ever!

Some more about symbolic networks

I've created some images to explain the ideas of the previous post. I think those ideas are very promising, since they indicate they could comply with the requirements/statements of previous posts (I can't academically make this statement until I have scientific proof of this, so it's a hypothesis):

• The energy equation between a biological network and a mechanical network should more or less hold, within a certain range and some research should be made if there needs/should be a factor in this equation.
  
• There shouldn't be an unroll to the functions that are called as part of the symbolic network
  
• There should not be an expectation of an output state/measurement (the network *is* the state and is always in modification)

The following is a PET scan that shows brain activity. Think of this as a screenshot at a certain point in time, when the network is processing some kind of thought:

You can see some areas of the network are totally unused, whilst others display high states of activity. Of course, it is very important to assess the brilliance factor and brilliance degradation/fallout (time it takes to decrease the brilliance) within the context of this picture.

The brilliance is basically activation of neighboring nodes. So thinking about one concept can also easily trigger other concepts. The "thread" of a certain context would basically guide the correct activation path.

I imagine a kind of network of symbols that are interconnected as the following picture:

The "kind-of" association is not shown here, because I'm not sure it really matters at this point. The "kind-of" assocation in itself can also be associated with a concept, that is, the "kind-of" can be an ellipse itself. So there is some loss of information in the above diagram, but that loss is not being considered at this time.

You can see that concepts are shared between other concepts to form a very complicated mesh network. It's no longer ordered in layers. If you consider the strength of an association (how strongly you associate something with something else) as the line that is inbetween it, then I could ask you: "What do you think about when I mention exhaust gas?". Then your response could be car or bus. The lines thus represent associations between concepts.

Wheels are known by both the concept car and bus. Also notice that this network is very simple. As soon as you gain expert knowledge in a topic, this network will eventually split up into sub-topics with expert knowledges about specific kinds of wheels and specific kinds of buses and specific kinds of cars and how they relate to one another. Generally, we distinguish my car and other cars, which is one example of a topic split. This statement of expert knowledge is derived from my little nephew looking at his book. For him, a motor bike, a bus, a cabriolet, a vw and things that look the same are all cars at this point in time. Later on, he'll recognize the differences and store that in memory (which is an interesting statement to make, as it indicates that this network is both a logical representation and association, but also memory).

The connections in this kind of symbol network can still be compared to dendrites and synapses. The strength of an association of one concept with another is exactly that.

Now, if you consider that you are reading a story and you have certain associations, you can also imagine that these concepts "fire" and are added to a certain list of recently activated symbols. Those symbols together form part of the story and the strength of their activation (through the synapse strength, their associations with other topics and a host of factors, basically what the network has learned) will in different contexts slightly change the way how the gist of that story is remembered.

If you store the gist of this list (produced by a certain paragraph or document), it should be possible to compare this with other gists through some clever mathematical functions, so that gists of one document can be compared with others. Gists are also methods of reducing storage details and storing it in a much compressed form.

Consider the final picture in this post:

It shows a simple diagram of, for example, what could be a very short children's story (well, we're discussing this text at that level basically). Dad goes home in his car and enters the house. He sits on the couch and watches the tele. If you remove the verbs of these statements, you'll end up with a small network of symbols that have some relation to one another. I feel hesitant to jot down the relationships between them in this network of symbols. I'd rather add some layer on top of these symbols that manipulate the path that a certain story or context takes. So, the concepts are always somehow related, but the thread of a story eventually determines how the concepts really relate to one another. Therefore, the thread manipulates the symbolic network in different ways.

So... what about design for an implementation? In game design, even when it was still 2D, the designers already started with large lists of events and lists of nodes for path finding for example. Between each frame, these lists were rebuilt and re-used to update the AI or action. Those design patterns should be reusable in this context:

• Start with reducing a text to its nouns only
• Process the nouns one by one
• For each noun:
• Reduce the activation factor of current concepts in the activation list
  
• Apply the synapse factor to the current noun
• Add concept to the activation list
• With a reduced activation factor by synapse, add related concepts that are connected to the currently processed concept to the list as well
  
• Get an inventory of the highest activated concepts in the list
• Store the gist list to describe the text
  

Obviously, the thing that is missing from the above is the intention, the thread. So a text that describes a guy in India getting of a bus to his house may equate to the same gist of a bus in San Francisco that happened to drive past an Indian restaurant.

So motivation and thread of a story is something entirely different from its concepts. Should this be part of the network?? In a way, I think it should be possible to think of it as a layered network above the symbolic network, a different kind of representation with links to the other network to describe actions and objects that are acted upon.

It's the state of the network, stupid!

Back to the discussion on Artificial Intelligence and Neural Networks. This blog hosted some meanderings and thoughts on AI and how this relates to NN. I read books from Steven Pinker and others regarding NN's and I understand how these work from a helicopter view (I even implemented some). I then analyzed that the perspective of NN's on AI is probably horribly wrong, if you compare biological networks against mechanical ones.

Computers were from the very start designed to work on input and then produce output. Output is a very binary-like state. It's either this or that, it's a range of numbers and the output is generally an exact number. There's not really a way to represent an answer that represents two (slightly) different states in one answer by itself.

This morning, I woke up and considered that this general approach that was taken as part of AI way back is probably wrong. Even though computers are supposed to produce output that can only be interpreted in one single way, the "output" of human brains doesn't really exist as output per se. I'm more thinking of answer and thought as a kind of "state of the network" at some point in time. The frequency of thought is given by the "frequency" of the network, although this seems a very weird term to use for biological networks. It's probably totally independent.

If you look at CAT-scans though, you'll see something interesting. Not all neurons are active at all points in time (very contrary to mechanical networks, which generally have all their nodes and parts connected, so that turning one part will turn and impact another). And the granularity of our analysis on the human brain is not at neuron level, but at the level where we see a general number of neurons receiving activity. So if one neuron A next to neuron B is fired, only A would be active, but B would not be assessed.

So it's like regions of interconnected neurons are active at one sweep, not the entire network. And there's no output like a machine, only a list of active and recently active neurons. Every sweep, the active list is modified and moved back into a ring of memory.

If we reconsider neurons as nodes in a network and replace them with symbols instead, we can probably come close to a logical representation of a thought network. So, a neuron by itself doesn't represent anything, but impacts something physically. A symbol is highly representative of something, but doesn't necessarily impact anything, it is only connected to other symbols.

The symbolic network is then like a neural network, only it works with nouns, with symbols. The symbolic network allows an infinite number of nodes to be added, as long as there exists a process to interconnect symbols as soon as there is a relation to be determined between them.

Now, imagine what happens and assume this symbolic network is properly connected. When you mention a word car, or the smell of exhaust gas, or a picture of a car, those symbols are activated. The joint activation of car, exhaust gas and the picture should activate a symbol of car (without annotation) as a concept, so that the network understands that car is being discussed.

If you now introduce a text with nouns and verbs and assuming the computer has grammatical analysis capabilities, you can process this text within the symbolic network and at the end of some paragraph / passage of text, the network has a certain state of activity. Some regions are highlighted and other regions are black. If you'd keep a list of symbols that are activated, then you could store that list (region) as a representation of the text.

So, the objective is not to store the text word for word, but to store the associations and context of the paragraph. Mentioning the words in a search term would probably produce the text again and the more aligned with the paragraph it is, the more likely it is to be found.

Memory is also important. There are actually different rings of memory in this model (short-term and long-term are too generic). Reading a passage would store the gist of that passage into a different symbol. The gist are basically those nodes that had the highest activation of a certain paragraph after the cycle is completed. So storing the gist of one paragraph with another may develop a description of a document that is highly descriptive. It's not necessarily the word that is mentioned, it's the concept and the relation to other symbols. It's possible that a symbol is highly activated that was not explicitly mentioned in the text.

The symbolic network is the representation of nouns of our language, but verbs are the activities, the manipulations in our mind of those symbols. It seems then that within the context of this blog post, the verbs correspond to real intelligence (which is not described in this post yet). The nouns are just perceptions and mapping them to symbols. Real thought, the one that creates (is creative) and can reproduce and come to certain conclusions is a totally different matter. That sounds like real artificial intelligence.

Project Estimation

There are some interesting discussions taking place on project estimation. Project Dune 1.4.2 came out yesterday, where you can register your estimates per activity per scope statement. So that's basically "ballpark" level of detail.

There are some new requirements that are coming up that look really interesting. PM's really seem to (want to) use some kind of math magic to get to certain numbers and then tweak them.

So, what really matters in a project? What are factors that, if they change, significantly affect the timeline of a project?

I've already mentioned that the level of communication is a huge issue. So, if your team increases, you'll increase contention within the team, the need for communication (which loses time) and so on. Especially at the start, where the PM actually intended to move fastest. So the rest of the trajectory people are generally told to "get on with it", leading to unsatisfiable levels of communication and lower standards of quality.

Oh well! :)... Expect some more notices in the future with regards to this topic.