Tuesday, February 17, 2009

Understanding the value of Data Mining

Artificial Intelligence is quite a large area, academically speaking. Truly accessible applications for business are not so common however. Data mining may be one of them, but as a manager you should approach data mining correctly. Some people may tell you that having a data warehouse is the first pre-requisite to be able to undertake the first steps in data mining. Other managers will tell you they've gotten "SQL in their fingers" and will be able to come up with some interesting discoveries given some experimentation time. Some lucky miners have access to a large data warehouse and just run a few queries to confirm suspicions or new theories.

The thing is, data mining is not like a hobby, it's a true profession. You'll know the difference between a hobbyist and a professional when you ask them how they think about data mining. Find out if they think it's about developing a hypothesis and then testing the data against it, or whether it's discovering new truths about data. True data mining is guided by business perspective. Let someone from the business tell you where it aches or where they want to improve and then go off to find their answers. Are customer leaving? Does the business want to become more efficient in one area? What is the knowledge they lack, rather than the knowledge they want to confirm?

Data mining, to be efficient, needs a focus. It's easy to tell a company that the only way to even consider commencing in mining is having a data warehouse. Time has proven that the establishment of such a DWH is time-consuming and doesn't provide the necessary payback because half is not used.

Factor in your business goals with these requirements. What do you want to achieve? Let an expert go over these goals and consider the data flows that are needed. Then just get those data flows from wherever you can get them, load them in a database (general ETL), don't necessarily model it out, especially if you need it only once, derive your conclusions and move on if it's not giving the payback you require.

Especially in these times, managers need to be more focused on the evaluation of what provides payback and what doesn't. Don't linger around, find out which things are promising, eliminate those things that cost money.

Data mining can be very strategic also for not so established companies. Once you gather sufficient volumes of data, you can start considering it, but you definitely need expert guidance here, it's not a job that the common software engineer can get away with.

Worst of all, don't rely on anyone offering "neural networks" to tell you about things. Make sure to use proper algorithms to smooth and massage your data, so it becomes more interpretable for human beings. Graph it out, since that's the best way to visualize complex data sets. One picture is better than 5,000 numbers.

And finally, it's very, very unlikely that any data mining algorithm will tell you: "Please do A in order to achieve B". In general, data mining results require interpretation and understanding, especially understanding about its limitations.

Thursday, February 12, 2009

Developing your custom kernel with qemu

I'm preparing a presentation that is hopefully going to be shown at an internal event at Sogeti. We're discussing the internals of Operating Systems. For this presentation, the intention is to compile a pre-created, very simple and basic kernel, which doesn't do much but print "Hello World!".

Well, if you're interested in doing kernel development yourself, there are plenty of resources around to help you get started. The not-so-easy thing is getting your development environment in order, so that you can actually run and debug the kernel. Here's a set of commands that help you do that. I'm using qemu as a VM emulator, gcc and nasm for development and Ubuntu as a host development system.
  1. qemu-img create myos.img 32M
  2. losetup /dev/loop1 ./myos.img
  3. fdisk -u -C65 -S63 -H16 /dev/loop1
  4. ( create a primary bootable partition across the entire disk)
  5. losetup -d /dev/loop1
  6. losetup -o32256 /dev/loop1 ./myos.img
  7. mkfs -t ext3 /dev/loop1
  8. mount -t ext3 /dev/loop1 /mnt/image
  9. df -Th
  10. losetup /dev/loop2 ./myos.img
  11. ln -s /dev/loop2 /dev/loop
  12. cd /mnt/image
  13. mkdir boot
  14. mkdir boot/grub
  15. cd boot/grub
  16. cp /boot/grub/stage1 ./
  17. cp /boot/grub/stage2 ./
  18. cp /boot/grub/e2fs_stage1_5 ./
  19. grub
  20. grub>device (hd0) /dev/loop
  21. grub>root (hd0,0)
  22. grub>setup (hd0)
  23. grub>quit
  24. vi /mnt/image/boot/grub/menu.lst
  25. menu.lst:
    timeout 0
    default 0
    title minimal­kernel
    root (hd0,0)
    kernel (hd0,0)/minimal­kernel
  26. rm /dev/loop
  27. losetup -d /dev/loop0
  28. losetup -d /dev/loop1
  29. mbchk kernel.bin
  30. cp kernel.bin /mnt/image/minimal-kernel
  31. umount /mnt/image
And then run it with:

qemu -m 100 -hda myos.img -boot c -no-kqemu

There you go! Of course, once the image is created and sorted, for development you only need to repeat step 30. You can remount the image in a build script:

mount -o loop,offset=32256 ./myos.img /mnt/image

Here are more resources:

Monday, February 09, 2009

Physical Symbol Systems

At the university, I'm now following a course on knowledge systems. There's plenty of room for one to be sceptical about knowledge systems, in the sense that it's possible to think of general procedural programs as knowledge systems too, the knowledge system is identified by the ability to separate knowledge from processing (the inference engine). This great distinction, if followed to the letter, means that knowledge systems are only valid once knowledge is explicitly declared and not mingled with source code of the inference engine. This rules out most procedural program implementations, if not all. In a simple interpretation, I'd say that knowledge systems can only be built in Prolog.

I don't want to discuss this item for the rest of my post however. What is more interesting is the hypothesis stated by Alan Newell and Herbert Simon (picture above). They stated:
A physical symbol system has the necessary and sufficient means for general intelligent action.
This implies a couple of things. First, that intelligence can be thought of as symbol manipulation. Second, since computers can be thought of as symbol manipulation machines, computers can in theory become intelligent.

Many efforts on representing knowledge in computers generally start with some written-down term of a particular symbol. Reasoning with text however is a bit daunting, that is, the symbol itself cannot be broken down and its meaning is invoked through its connections with other symbols and concepts.

Thinking very abstractly however, symbols are representations of knowledge, and symbols need not be visual. I could for example describe a cow as 0x7A832B12 and add further different representations of the same thing. Colloquially, I could add the image of a cow, the sound and the smell and tie them together as representations of the same thing.

Symbol reasoning systems then require the computer to categorise the symbols themselves, to find ways they are equal, ways they are different and how symbols might be related to other symbols and in which way. It's even possible that the relation itself is yet another symbol.

A limitation of our thoughts might be that we rely on our language too much in order to be able to debug knowledge systems. I'm considering that language itself might not be the most efficient way to develop reasoning systems.

Another important quote that I read today is that there is no true way yet to assign meaning to symbols, however meaning can be represented in a computer, that is.

Hmm.... the ideas I have about this subject are really abstract and it's almost impossible to write them down in a sensible way, at this time. But why rely on our or any other language for a representation of our knowledge? If knowledge has different appearances, then shouldn't we let a computer choose how it decides to store it? The designer of any program makes full decisions about data, and data structures.

For learning machines though, we may need to innovate on how data and knowledge is stored, such that more complex systems could possibly use it in different ways, hopefully with the ability to derive new knowledge from existing knowledge, which seems a current hard limitation for a computer at this time.

So... goal: Build a knowledge system without a specific design for the storage of knowledge, or built it as a hybrid combination of implicit knowledge with explicit knowledge.

Monday, February 02, 2009

The world explicitly in "math vision"

Computers don't have sense of anything. They're basically only good at processing according to a couple of defined guidelines (the program) and nothing much else really. It's just a processor, like a blender processes food and was designed to do so.

Artificial Intelligence is so interesting, because it comes up with novel ways to reason with input. In a very narrow (or is it broad?) definition, any computer program is artificially intelligent, because any program uses "if-then" rules. However, engineers typically do not accept that definition, because the intelligence conveyed by such programs are not surprising and do not supersede our own capacity for reasoning. Popularly speaking, being intelligent means that someone, some animal or something is behaving in a way that surprises one.

A (regular) program cannot execute other rules than those "if-then" rules it has been designed to handle. That generally makes it very explicit and consistent in its behaviour (unless bugs are in the program). Let's assume that any program I'm describing here has been 100% tested and is guaranteed 100% bug-free.

The world to physicists and mathematicians looks different in many ways. There's a constant awareness of approximations of behaviours through formulas and the awareness that some problems that look ridiculously simple are astoundingly hard to solve or describe mathematically.

For A.I. to continue into its own field, it'll be an ongoing battle to get the computer to reason and "understand"? (this latter term should be used very carefully) its environment better. Roger Penrose highlighted four different viewpoints on the mind (mind != brain), where on one extreme the mind is 100% mystical and unexplainable and at the other extreme it's 100% computational. Dr. Penrose is a physicist and he doesn't seem to be inclined to believe that the mind is 100% computational (viewpoint A), but there's a strange missing link that allows computational processing machines to become aware (although one could also argue that awareness actually means introspection abilities), the state vector reduction between quantum physics and that of classical physics.

When one mentions "make the computer smarter", one generally assumes that the computer should become more like us.... but thinking about it... there's no reason why it must or should. I've argued before that humanity is pretty arrogant when it comes to words about intelligence and it basks in the light of its own narcissistic tendencies. Consciousness is not truly a pre-requisite for life-like intelligent action. Although consciousness itself is very likely not achievable in non-biological machines... can some sort of consciousness be simulated or modeled?

Some posts back I wrote about rule mining. For a computer to simulate consciousness, it should be able to deduce new rules, descriptions by analysing its perceptions. However, in order to even start doing that, it must see the importance of doing so in the first place. And in order to see the importance, it must understand the context and environment. So this is a circular argument, seemingly? Well, we certainly don't get born with objectives and understanding from day one. So there's a learning element involved and impulses that determine our goals. Baby's do have certain goals, although simple: "eat, poop, sleep" and they'll keep on crying until the goals are satisfied. Babies are human, but popularly, we're not considering them conscious yet. Possibly we start considering little children conscious when they start talking...?

Here's an account of a professor who has autism. The read is very interesting, especially the last summation, where four different levels of consciousness are given:
  1. Consciousness within one sense.
  2. Consciousness where all the sensory systems are integrated.
  3. Consciousness where all the sensory systems are integrated with emotions.
  4. Consciousness where sensory systems and emotions are integrated and thoughts are in symbolic language.
Next to the brain, we're also responding to chemical changes in the brain. In fact, just as the ear receives auditory information and the eyes receive visual information, we could think of chemicals and proteins to produce chemical information for our brains to process. Those proteins and chemicals indicate our own state to ourselves, besides the faster processes like pain (if pain were transmitted through chemicals, it'd probably take at least 20 seconds before a response occurred, not efficient!).

Quite some time ago, I mentioned that emotions are the driving forces behind humanity. What I really meant was that without emotions or feelings, we won't feel any urge to start doing anything. It's like a computer on a desk with 0% CPU usage, 0% disk I/O. Only when the proper impulse is given, is the goal generated and will we start to find ways to achieve those goals.

So it naturally follows that a 'conscious' computer should have goal-generating abilities to function more like an animal. The problem here is that one doesn't just code a "goal-generating algorithm". Different people pursue different goals. It depends on experience, outside stimuli, upbringing, different chemical compositions, talent, preference... So it's something that more or less 'grows on you'. How can the same thing be grown in an A.I.?

At a lower level... the building blocks that make US tick and develop the preference in the first place... what is it? I mean, what actually did develop consciousness or shaped it into being? If we assume babies are not conscious, then something at a lower level is developing/placing something there.

This suggests a lower level of being that is using consciousness as a tool for achieving its ultimate goal(s). That goal may be very simple (survival? pro-creation? endless aim for better well-being?), but through our conscious "processing"? layer, it translates this into several different sub-goals.