Something to watch: FACETS

FACETS is probably one of the most interesting research projects that's taking place right now. Well, together with the LHC :). Previously, I posted on the architecture of mind. That post resembles the envisioning of a totally different kind of hardware as it would apply to (re)modeling the human brain. Well, the people at heidelberg uni are now doing this. They're constructing hardware that uses a radically different design than micro-processors. From their site:

By creating specialized digital hardware processors it might be possible to gain an advantage over microprocessor-based systems. Still, it is unlikely that this will be more than an order of magnitude, since they are based on the same technology as microprocessors: The neural circuits remain to be realised with numerical solutions of differential equations. The biggest problem lies in the fundamentals of Moore's law itself: the scaling of process technology. In the current semiconductor roadmap the progress has already slowed down. The transistor density of high-performance microprocessors is likely to increase only by a factor of 25 from 2004 to 2018. A power consumption of 300 Watts is predicted for such a hypothetical chip while the on-chip operating frequency will be in the 50 GHz range.

I've blogged before on consciousness and how this might relate to computers, and what a design could look like. And some more. And more.

Changing the hardware is extremely important and much more likely to become successful.

A large problem is still the coordination between things and the problems related to associative memory. That is, when we say "cat", we instantly recall associations related to the word, strongest first (black? kitten? dead mouse?).

Most explicit knowledge systems scan their entire memory base, or have otherwise explicitly defined boundaries around knowledge to hierarchically exclude certain pools of knowledge programmatically. Thus, a key pointing to some piece of information is not recognized as such and doesn't cause a specific part of memory to highlight. It's requiring a sweep of memory to see with which asset of knowledge it's associated.

In order to be successful in the future, I think it's necessary to find ways to prevent that, to directly find some location/pool/hierarchy where something is probably located, such that it finds a match or has the ability to locate it together with other like members.

Some more about symbolic networks

I've created some images to explain the ideas of the previous post. I think those ideas are very promising, since they indicate they could comply with the requirements/statements of previous posts (I can't academically make this statement until I have scientific proof of this, so it's a hypothesis):

• The energy equation between a biological network and a mechanical network should more or less hold, within a certain range and some research should be made if there needs/should be a factor in this equation.
  
• There shouldn't be an unroll to the functions that are called as part of the symbolic network
  
• There should not be an expectation of an output state/measurement (the network *is* the state and is always in modification)

The following is a PET scan that shows brain activity. Think of this as a screenshot at a certain point in time, when the network is processing some kind of thought:

You can see some areas of the network are totally unused, whilst others display high states of activity. Of course, it is very important to assess the brilliance factor and brilliance degradation/fallout (time it takes to decrease the brilliance) within the context of this picture.

The brilliance is basically activation of neighboring nodes. So thinking about one concept can also easily trigger other concepts. The "thread" of a certain context would basically guide the correct activation path.

I imagine a kind of network of symbols that are interconnected as the following picture:

The "kind-of" association is not shown here, because I'm not sure it really matters at this point. The "kind-of" assocation in itself can also be associated with a concept, that is, the "kind-of" can be an ellipse itself. So there is some loss of information in the above diagram, but that loss is not being considered at this time.

You can see that concepts are shared between other concepts to form a very complicated mesh network. It's no longer ordered in layers. If you consider the strength of an association (how strongly you associate something with something else) as the line that is inbetween it, then I could ask you: "What do you think about when I mention exhaust gas?". Then your response could be car or bus. The lines thus represent associations between concepts.

Wheels are known by both the concept car and bus. Also notice that this network is very simple. As soon as you gain expert knowledge in a topic, this network will eventually split up into sub-topics with expert knowledges about specific kinds of wheels and specific kinds of buses and specific kinds of cars and how they relate to one another. Generally, we distinguish my car and other cars, which is one example of a topic split. This statement of expert knowledge is derived from my little nephew looking at his book. For him, a motor bike, a bus, a cabriolet, a vw and things that look the same are all cars at this point in time. Later on, he'll recognize the differences and store that in memory (which is an interesting statement to make, as it indicates that this network is both a logical representation and association, but also memory).

The connections in this kind of symbol network can still be compared to dendrites and synapses. The strength of an association of one concept with another is exactly that.

Now, if you consider that you are reading a story and you have certain associations, you can also imagine that these concepts "fire" and are added to a certain list of recently activated symbols. Those symbols together form part of the story and the strength of their activation (through the synapse strength, their associations with other topics and a host of factors, basically what the network has learned) will in different contexts slightly change the way how the gist of that story is remembered.

If you store the gist of this list (produced by a certain paragraph or document), it should be possible to compare this with other gists through some clever mathematical functions, so that gists of one document can be compared with others. Gists are also methods of reducing storage details and storing it in a much compressed form.

Consider the final picture in this post:

It shows a simple diagram of, for example, what could be a very short children's story (well, we're discussing this text at that level basically). Dad goes home in his car and enters the house. He sits on the couch and watches the tele. If you remove the verbs of these statements, you'll end up with a small network of symbols that have some relation to one another. I feel hesitant to jot down the relationships between them in this network of symbols. I'd rather add some layer on top of these symbols that manipulate the path that a certain story or context takes. So, the concepts are always somehow related, but the thread of a story eventually determines how the concepts really relate to one another. Therefore, the thread manipulates the symbolic network in different ways.

So... what about design for an implementation? In game design, even when it was still 2D, the designers already started with large lists of events and lists of nodes for path finding for example. Between each frame, these lists were rebuilt and re-used to update the AI or action. Those design patterns should be reusable in this context:

• Start with reducing a text to its nouns only
• Process the nouns one by one
• For each noun:
• Reduce the activation factor of current concepts in the activation list
  
• Apply the synapse factor to the current noun
• Add concept to the activation list
• With a reduced activation factor by synapse, add related concepts that are connected to the currently processed concept to the list as well
  
• Get an inventory of the highest activated concepts in the list
• Store the gist list to describe the text
  

Obviously, the thing that is missing from the above is the intention, the thread. So a text that describes a guy in India getting of a bus to his house may equate to the same gist of a bus in San Francisco that happened to drive past an Indian restaurant.

So motivation and thread of a story is something entirely different from its concepts. Should this be part of the network?? In a way, I think it should be possible to think of it as a layered network above the symbolic network, a different kind of representation with links to the other network to describe actions and objects that are acted upon.

It's the state of the network, stupid!

Back to the discussion on Artificial Intelligence and Neural Networks. This blog hosted some meanderings and thoughts on AI and how this relates to NN. I read books from Steven Pinker and others regarding NN's and I understand how these work from a helicopter view (I even implemented some). I then analyzed that the perspective of NN's on AI is probably horribly wrong, if you compare biological networks against mechanical ones.

Computers were from the very start designed to work on input and then produce output. Output is a very binary-like state. It's either this or that, it's a range of numbers and the output is generally an exact number. There's not really a way to represent an answer that represents two (slightly) different states in one answer by itself.

This morning, I woke up and considered that this general approach that was taken as part of AI way back is probably wrong. Even though computers are supposed to produce output that can only be interpreted in one single way, the "output" of human brains doesn't really exist as output per se. I'm more thinking of answer and thought as a kind of "state of the network" at some point in time. The frequency of thought is given by the "frequency" of the network, although this seems a very weird term to use for biological networks. It's probably totally independent.

If you look at CAT-scans though, you'll see something interesting. Not all neurons are active at all points in time (very contrary to mechanical networks, which generally have all their nodes and parts connected, so that turning one part will turn and impact another). And the granularity of our analysis on the human brain is not at neuron level, but at the level where we see a general number of neurons receiving activity. So if one neuron A next to neuron B is fired, only A would be active, but B would not be assessed.

So it's like regions of interconnected neurons are active at one sweep, not the entire network. And there's no output like a machine, only a list of active and recently active neurons. Every sweep, the active list is modified and moved back into a ring of memory.

If we reconsider neurons as nodes in a network and replace them with symbols instead, we can probably come close to a logical representation of a thought network. So, a neuron by itself doesn't represent anything, but impacts something physically. A symbol is highly representative of something, but doesn't necessarily impact anything, it is only connected to other symbols.

The symbolic network is then like a neural network, only it works with nouns, with symbols. The symbolic network allows an infinite number of nodes to be added, as long as there exists a process to interconnect symbols as soon as there is a relation to be determined between them.

Now, imagine what happens and assume this symbolic network is properly connected. When you mention a word car, or the smell of exhaust gas, or a picture of a car, those symbols are activated. The joint activation of car, exhaust gas and the picture should activate a symbol of car (without annotation) as a concept, so that the network understands that car is being discussed.

If you now introduce a text with nouns and verbs and assuming the computer has grammatical analysis capabilities, you can process this text within the symbolic network and at the end of some paragraph / passage of text, the network has a certain state of activity. Some regions are highlighted and other regions are black. If you'd keep a list of symbols that are activated, then you could store that list (region) as a representation of the text.

So, the objective is not to store the text word for word, but to store the associations and context of the paragraph. Mentioning the words in a search term would probably produce the text again and the more aligned with the paragraph it is, the more likely it is to be found.

Memory is also important. There are actually different rings of memory in this model (short-term and long-term are too generic). Reading a passage would store the gist of that passage into a different symbol. The gist are basically those nodes that had the highest activation of a certain paragraph after the cycle is completed. So storing the gist of one paragraph with another may develop a description of a document that is highly descriptive. It's not necessarily the word that is mentioned, it's the concept and the relation to other symbols. It's possible that a symbol is highly activated that was not explicitly mentioned in the text.

The symbolic network is the representation of nouns of our language, but verbs are the activities, the manipulations in our mind of those symbols. It seems then that within the context of this blog post, the verbs correspond to real intelligence (which is not described in this post yet). The nouns are just perceptions and mapping them to symbols. Real thought, the one that creates (is creative) and can reproduce and come to certain conclusions is a totally different matter. That sounds like real artificial intelligence.

Modularity of Mind

Merry Christmas everybody. I'm just writing up some recent thoughts.

Some books I was looking at with Amazon consider the mind as a thing with a modular composition. A module for language, another for reasoning, and so on. Logically it may be possible to dissect it this way, but I don't think it should be confused with physical modularity so quickly.

The previous post considered pattern recognition as the main topic of reasoning. I thought about this more and more and I just felt as if something else is missing. Pattern recognition is all around us and necessary, but it just doesn't feel like AI and neural networks are the beef of what our minds are about. I miss something that constitutes logic. Because, even if we have the ability to recognize words from a stream of noise, visual patterns in what we say or even objects and so on, it does not yet allow us to manipulate those things and combine them with other items.

Or, in other words... In my meanderings I missed the element of "consciousness", what it is about and how it is intertwined within our abilities to recognize patterns. I also think of consciousness as the ability to learn, identify and establish new patterns. For, in order for an artificial network to learn things, something must exist that compares output with input and recalibrates the network. What is that thing inside our mind?

An easier way to think about this is skill-acquisition. When we learn to drive a car or a bike, we combine certain inputs together (balance, sight, motorics, accuracy, action/consequence patterns, danger recognition) and eventually patterns are created, which allow us to 'automatically' perform the task. Before it gets there however, we are consciously accompanying each action and consciously making adjustments until we finally get it. So it feels as if besides pattern recognition that is as some kind of auto-pilot, we consciously need to evaluate the world around us to learn from it. And even then, we consistently apply consciousness throughout a journey, for example when encountering new territories or when certain elements have changed or when traffic is significantly dense. (It is next to impossible to execute other tasks in those events).

So I am basically concluding that pattern recognition by itself is not sufficient for the human mind. But I would not go as far as saying that the mind can be thought of a physically modular kind of thing. I'd rather think of it as a richer neural network than AI constitutes, probably something that still contains other elements for reasoning, logic and learning that we are yet unable to perceive. Memory (and retrieval) is another thing that I started to neglect.

The symbols that flow through the network may not be numbers. But if these are not numbers, what are they? If I consider an AI network in computers that does not use numbers, but keys or some gibberish that I make equivalent to some kind of symbol, will the output product be a sensible product after the network has manipulated and processed it? It sounds too random for that to be true, unless the output product is somehow matched to something else. Maybe these outputs are basically non-sensical symbols that are keyed to some kind of knowledge. Whereas knowledge in AI networks are embedded into the weights, I think of knowledge slightly differently when applied to neuro-science.

Architecture Of Mind

Previous posts discussed many individual things of the inner workings of the mind that I have read so far. As a SW architect, I prefer pictures over words to convey meanings. I've been working on a picture that is reminiscent of the OSI layer in Computer Science, which describes how communication takes place over the Internet. The picture is here:


Based on the books, it shows that at the physical layer, we have neurons and chemicals that are somehow interacting together. Much like a computer, this layer only becomes interesting for absolute experts. It is at such a low level that understanding how it works might make clear how things operate, but since it is so complex and executes at a very high frequency and most probably is parallel, it is difficult to develop expectations on that level on operation, unless you subdivide the most basic functions (CPU instructions) into grouped functions with a particular purpose. And further generalization of that.

In this picture, I see neurons as transistors or silicon. So they are basically thousands and thousands of black boxes that interoperate together and the sum of all its minuscule operations have a certain effect. The silicon accomodates the flow of currents for communication between transistors, where transistors modify that flow. The same is probably true for neurons and synapses. This is about where the direct analogy (should) stop(s), as I believe that the current household computer as we know it has a totally different architecture than the human mind due to its requirements on determinism and finity. The human mind could be inifinite (?) in its capability to produce new thoughts and new goals, based on previous contexts. Computer programs are mostly single-goal oriented and are generally not engineered to produce new goals along the way.

The physical layer thus contains neurons (or "mush") that accomodates emotions, feelings, the mind's eye, pattern recognition, rotation, language?, reasoning and thus intelligence. Intelligence can probably also be described as the interaction between pattern recognition, reasoning (which is setting new goals or imagining consequences (developing expectations) based on previous experiences). Inventions and innovations are the acts of making new associations where previously there were none. With each new created association, we are likely to become more intelligent. Ones "aptness" to develop associations is probably ones IQ. Emotional Intelligence is probably the sympathy in observing one's behaviour and developing expectations based on those observations through other associations and one's aptness to read behaviour (be attentive to signals) and so on.

In the previous post it was said that the main goals of our being are determined by our emotions and feelings. Feeling hungry means instinctively searching for food that is edible. If food is not around, it is invoking our intelligent system to determine the best course of action to get it (or zero the emotion if effort is larger than desire). If you "feel" lazy, you might want to buy something at the local gas station rather than go to the supermarket. So the emotional system works very close together with the intelligent system to resolve the problems. If you have access to a car, you might want to go into town, park the car and do some shopping. Unless you forgot where you put your car keys, in which case it might cause frustration and the setting of a new goal by taking the bike. Unless it is perceived to rain outside and the feeling of wetness is not a very pleasant foresight, which might cause you to look into the freezer and defrost some ready-made meal instead after all, even though initially the goal was looking forward to something fresher. The interaction of emotions, feelings and intelligence is clear. The discovery through our intelligence and memory that something is impossible might cause feelings of frustration, which might release some chemicals that cause our brain to go into a higher state of awareness, much like adrenaline that prepares the body for a fight.

The main goal in this case is always set by emotions or feelings. I haven't yet thought deep enough to find cases where a main goal is 100% determined by intelligence. Intelligence can set subgoals to achieve the main goal. The subgoals are determined by imagining how the main goal can most efficiently be achieved, which is done by looking for a path to that goal. This is mostly done by looking at historic events and how well this worked out in the past. If we engage a barrier that blocks access to the main goal, we invoke our associative mind and reasoning and expectations of outcome to try to remove that barrier. A sense of urgency might change how we reach those goals.

So reasoning and assessments highly require the services of the mind's eye, imagination and memory. The logic revolves around imagination and the analogy (pattern similarity) of other outcomes. The main goal determines our eventual behaviour always in the long run for the event. Subgoals might change our behaviour slightly or modify it entirely temporarily, but should always serve the main goal in the long run.

The picture is not quite complete as there is closer interaction between the emotional system and behavior. One can imagine that our body has been programmed to react instinctively to one person's behavior, a direct response instead of an evaluated response by the mind (if not, we would probably look and act like robots).

There is a continuous interchange between the emotional system and the "Intelligence" system. The "depth" of the recursion in the intelligence system (the ability to resolve complex cases like "if this, then that, and then that, however when, if not, etc."), the congruity that particular mind needs (or incongruity it can suffer) to find associated material in memory and the amount of experience are probably the best factors that determine intelligence.

As I have said in previous posts, I imagine that the brain is thus not a 'stack-based' computer, but a machine that always moves forward within its own context. I also imagine the context as something that is fluid rather than hacked in stone (as is the case with computers). The following picture shows how I regard memory, which I call "associative memory", since it recalls symbols as we listen to a piece of music, hear speech, see a scene or "think about" / "imagine" things.


The picture shows a line, the "thread" of a conversation or the "thread" of an observation/thought. We have the interesting capability to "steer" our thoughts into new directions. Are we modifying or creating goals at the same time?

The items in associative memory are not just in "drawers" or memory locations like in a computer. A computer may use lists, linear memory or hash keys to organize information. But no matter which method is chosen, there is always an incremental cost to look things up. This cost is expressed in O-notation and in general, the more information you store, the higher the cost. This has the nasty side-effect that becoming smarter means becoming slower and in certain cases, some problems become unresolveable unless you work together with many machines. Google is one perfect example, where the system basically stores the information on the Internet. It uses many, many, many, many computers to open up that information to others. However, Google cannot "reason" with that information, it can however process it and modify its relationships for a particular purpose.

So the model above shows a kind of memory that does not store information and make it accessible through keys. It shows memory where elements are naturally associated with one another and where these associated elements are automatically brought forward. The "thread" determines the direction of the context and how further associations are made. The further away from the thread, the lower the activation of that particular synapse or memory element. The context is thus basically the elements that were invoked and what we know about them. The context (associations) also give us rules. If a certain (new) association cannot be made, the thread must be redirected or halted and a solution suggested by the reasoning part of the brain. when reading nonsense, we cannot allow the nonsense to be stored as reality in our brain, since that taints our model of the real world. How do we prevent this from happening? By testing the thread against our current model and see how it complies. Changing a belief then means changing certain associations that we took for granted.

It is also possible that strong beliefs are formed when certain associations are often walked by threads. So associations probably have weights. It is not uncommon for us to believe something very strongly as being associated, but then discover that the association is invalid. We resist breaking the association very strongly, because the new evidence is still a very weak association that does not have many associations with others. Only when we forge other associations with the new evidence do we accept it taking the place of another element. Whereas we don't just "forget" the other element either. It becomes like a ghost image superimposed on the initial association, which is tagged as a false belief.

A computer can thus not replicate this behaviour easily. There are constructs like linked-lists and so on, but linear memory (the way how memory is developed now) is not ideal as a storage element for associative memory. It would be easier to think of associative memory as elements that are somehow forming tiny threads between them (which can strengthen into cables) and probably move closer together.

It's difficult for many human beings to do two things at once. To think two things at once. This suggests we have the analogous single CPU available to us. Other research shows that when we process information, we can only deeply focus on 3-5 pieces of information and derive results from those. That is analogous to a CPU that has about 4 registers available for processing. However, Intel processors work intensely with stacks, which suggests that things unwind and continue. I think of the mind as a CPU that is always moving forwards, does not have a stack and just finds new goals and conclusions and stores them as associations in memory. In computer lingo, functions that return parameters or allow output parameters or pointers simply do not exist.

The point of this whole post is to reason about the architecture of mind as if it were possible to build it into a computer. I somehow see points in the logical function of the mind that are incompatible with the current Intel architecture I am familiar with. Memory is linear, but should be associative and fluid. The computer/OS is stack based and always "attempts" to resolve subgoals at that point in time. I think that stack-based computing is the barrier to further intelligent systems. These intelligent systems are difficult to imagine, because we are not familiar with them. Its architecture needs to be thought out. Maybe it becomes easier in the long run than deterministic systems (who knows?), or on the other hand maybe they are significantly harder to program. That can be expected when you model the human mind to some degree. But full artificial intelligence (reasoning systems) require associative memory that can forget, the ability to form (new) goals based on things perceived on the outside and so on.

Another thing on the "mind's eye" that I find incomplete... Some psychological or "HR knowledge" states that some people are "auditive" or "emotive". These are related to our senses like vision, gustation (taste), olfaction (smell), auditory (hearing) and somatosensation, where the last one is a fancy one to describe everything we sense in the body (allow me to include "emotional state" into that sensation as well). The mind's eye however has been described as a purely visive operation. However, I don't know whether someone has ever done research on how blind people for example would handle their "mind's eye".

I can also personally reflect on the "mind's eye" myself. It is basically imagination itself. But I cannot only imagine visible elements (images, which is a word that probably invoked this whole oversight), but I can also imagine auditory elements, music, feeling a hot pan, feeling a cold pan, smelling grass, smelling strawberry and even more... I can store those senses as elements in memory. So, rather than thinking about memory as a set of images, it's a set of experienced senses at some point in time that are inter-related and give me a more complete picture of some event or thing. When I store the element of "feeling intensely happy" with the image of freshly cut grass and especially with the smell of that, it is not difficult to imagine that somehow the smell of freshly cut grass in the future can evoke the same feelings.

That latter part somehow suggests that our mind works even more intricately. It is as if the "processed" elements of information that we store in our memory (not signals itself, but perceived signals, post-processed by our organs like the nose etcetera, the very signals that are sent to the brain for further processing), when these post-processed elements are recalled in our imagination, the recall of that item causes our senses to relive the stored event. Some research I read at some point in time stated that when we speak, our brains temporarily reduce our auditory senses, so that we can recognize our own voice. If not, we would be startled everytime we make noise as we'd think a stranger is in the room. Maybe this mechanism is more intricate and we're actually reusing those processing parts of the brain for reasoning itself and the brain is not just one big mush (or maybe just in the physical sense, but not the logical sense). We'd actually have dedicated areas that we can reuse in our imagination as well.

So I make the point that the "mind's eye" sounds incomplete and that the only way that we can make deductions and reasoning are through previously experienced things that are stored in memory, not only as images, but also as smell, auditory information and anything else we can perceive about the situation.

It would also make the case that a computer cannot become "aware" unless it is given multiple senses itself. A sense for emotion would be very difficult, as emotion is entirely internal and I have doubts it can ever be re-engineered (it doesn't look like anything). We could perhaps simulate it by perceiving behaviour. But given audio, images and things that we can externally observe, maybe it's possible to store things together and in the future build a computer that is capable of doing similar things with that information through associative memory and the development of a context in which the observations are assessed and reasoned with.