Tuesday, February 13, 2007

Carnaval in Recife

It's getting carnaval again. Starts this week. On Friday, everyone will power down the machines and they start a party in front of the building.

The company has a "bloco de carnaval" called "Dá O Loud". It's a joke between making a lot of noise and "download".

Saturday, Recife organises "Galo da Madrugada" (see picture). Imagine 1 - 1.5 million people on a small island... it's quite insane really. Anyway, I'm going to a different kind of carnaval more in the country-side (on sunday). It's in Bezerros and should be quite good.

The other days? I probably would work a bit on personal projects, take it easy, go to the beach, play some video games? etc...

Tuesday, January 30, 2007

Project Dune now stable...

Project Dune went stable yesterday. It still needs a detailed customization guide and a user's guide to be really complete, but the main beef is there for people to use and to provide feedback on.

The next steps for this project are aimed at separate modules for SCRUM project management and document management. Naturally, those modules will have certain breakout points as well to allow integration with Project Dune.

The project itself will slow down in new features and aims to resolve issues that the user community will (or should) raise. Then with some more stability under it, we can take it to other levels later on.

Friday, January 19, 2007

It's not all the same kind of development

I have had the opportunity to work on many different kinds of projects, and you cannot treat software development for embedded hardware the same as web server application development, not by any long shot.

Embedded software, operating systems, low level applications are 70-80% preparation in design, memory layout, memory management, documentation and planning. In contrast to high-level languages used on webservers, etc. The more low-level and smaller you go, the better you need to plan the software, since the tools available (or rework) to modify changes is going to be more difficult. And the lower you go in the system, the more other systems built on top are likely to be severely affected.

So, 70-80% planning in embedded systems with not a lot of coding (just make sure you *do* it right). This is against 20-40% of planning for application software, but requiring a lot of coding effort. The high-level languages have plenty of libraries available and the development tools help significantly with refactoring nowadays. Planning beforehand for the software is actually a risk factor, due to changing requirements from the clients and changing insights along the development process, or a change in the way how things could/should be done.

Therefore, one could "perceive" embedded software development as a "slow" process, but this is not truly the case. A design is in itself a produce of intellect, but it does not immediately convert itself to a materialized ability to the end user. Nobody however can deny that a good design produces immense benefits in the long run for a certain operating system or function.

The final argument has to do with update cycles. Embedded software *has* to be correct. Webserver or PC software can be updated (should not, unless for new features), but we have sort of generally started to accept that for PC software you produce a version that you think works and then continuously update the software after it is released. Everybody does it, so why not you? With embedded, this is simply impossible, since the user does not generally have the ability or skills to do this. And in general we tend to think of a product as "bad" if they have to return it to the shop for 'an upgrade'...

Thursday, January 18, 2007

MS Vista vs. Linux source code control...

MS Vista has been delayed a couple of times and this is often attributed to different factors. This article shows how Windows did the version management. Check it out:

You have a very large number of branches that sometimes need to be integrated. The whole point of these branches is to establish "quality control exit gates", where the code will not pass upstream unless it passes the quality control check. This is a nice thought and that part might work, but there is another consideration for quality to be made here... immediate synchronization with other code. Now, the person writing this article is convinced that without this level of "management", nothing will ever ship.

Consider the Linux kernel. There is basically one mailing list and one mailing list only. There exists an idea of "separation of expertise", but not really "separation of concern or by branch". This does not mean that branches are not used, but they are temporary. The only longer-lived branches are kernel version branches (2.2, 2.4, 2.6), so you have different streams there.

The mailing list is considered total chaos (if you live up to MS terms). Everybody has access to every comment, even if they don't work on it and comments on everything. Filtering on your area of interest may be quite difficult, but in the end you can gather insight into everything that is going on. Messages in the middle establish "code freezes" (sent by Linux) when another version of Linux is to be closed off.

What I am ignoring so far is that Windows is much more than just a kernel, so the comparison is not truly fair. The problems that Windows may have faced in Vista is not necessarily "quality control", but immediate access to new features and managing the streams of code in real-time. Now, I am not saying there is a solution to this dilemma, but there is a problem to be solved.

If you rely on the quality gates, then a new feature that you depend on may only arrive after two weeks. Forget about putting pressure. Then you need to start a political fight with another department with a different budget and different bosses to get this feature in or *your* project will delay. Truth is that it's not truly the concern of that other department.

Open Source does not have a deadline and no requirement to make everything backwards compatible. Windows this is highly desired. So Windows needs to maintain backwards-compatible API's using a new structure of the kernel, but "looking" the same. We all know this is an impossible task...

So... we can argue... is the shipping of Windows Vista truly a miracle and good management, or could it have shipped way earlier if they would have managed things differently? Their personal experience shows that the way how it initially worked wasn't actually working very well...

SCRUM

There are different methods for project management besides the "Project Management Body Of Knowledge". The ancient "accredited" methods put a single guy as the responsible person for finishing and executing a project. This is just an observation at this point.

"Scrum" is another way of "managing". Actually, it is not really management, but mostly delegation. The scrum method is derived from agile project management and the team becomes responsible for its planning, estimation and execution. Well, the team led by the scrum master, sort of like a quarterback.

What I like about the method is that it allows you to extract information about the daily progress. For example, each task that is defined in this method will be estimated for a single day and one day only. There are no tasks that take two days. If there is one, it will be broken down into two parts. This facilitates management and tracking.

In the regular project management, a task may sometimes take 3, 5, 7 or 20 days. In the minds of some developers, this gives them "slack" and sometimes too much slack that the time is not properly controlled. At the end of the task, it may either be finished or backlogged. Imagine the surprise... And in steps the project manager to negotiate with the client.

Having it broken down in single tasks really facilitates tracking. The cool thing here is that the "whip" is no longer in the hands of the project manager. There is a public board on the wall that shows the tasks to be done, who is working on what and there is therefore transparency on other people's performance. This sort of means that everybody in the team has an invisible "imaginary" whip. Even though you wouldn't publically make anyone accountable, the people involved perceive a certain team pressure for complying with their agreed tasks and feel thus pressure to finish. This makes the integration within the team a lot stronger if the tasks *are* completed and opens up an interesting discussion if tasks are *not*. (this is where people management comes into play).

In the context of "Project Dune", I am planning to build a separated module for this type of management, so that electronically it is possible to track projects this way. There are quite a number of manual tasks involved still that should be automated.
(feature point management, project burndown, product burndown and impediment burndown).

The whole essence of scrum is making a sprint of 4 weeks and putting your foot down to complete it and manage your difficulties (impediments) plus tasks on a daily basis. This really helps to integrate your team better and establishes a much nicer team atmosphere than other management methods can do. However, this does depend on the commitment and attitude of the team.

Wednesday, January 17, 2007

More on OS development...

I'm just playing around with the toy OS and learning quite a lot about how a computer works and what the kernel for a computer really has to do. I am not even working on device I/O or advanced features at all yet. Just memory management.

Memory management is subject to the following considerations:
  • Where to put the kernel code and data (you do not want to overwrite this)
  • Where to put the GDT and IDT
  • Where to put the memory "page directory" (full 4GB directory == 8 MB)
  • The ratio between kernel space vs. user space (linux = 1:3)
  • How to implement an optimal "malloc" routine
  • Page swapping (but not required, a kernel may also fault)
  • Optimization of the above requirements, like is done in Linux > 2.4
  • Memory page sharing through copy-on-write for example (by default the process shares the same code + data, but on any attempt to write to the page, the kernel makes a copy of that page)
So... great... lots of considerations. I am just at the part of where to put everything. My decisions haven't been ideal yet, because of boot-up limitations mostly. But slowly I should be able to work towards page allocation for a process, assigning it to a process and then subsequently the ability to run a certain process (albeit hardcoded in the kernel itself for the time being).

As said before, this is not going to be anything big, but will greatly assist in understanding other OS's a lot better. Maybe even I could use this knowledge to jump-start another project for a limited device and make the machine do something really specific in an easy way.

G>

Monday, January 15, 2007

How to start writing your own OS

Writing a fully fledged Operating System is not a task that can be finished by a small group of people, but it is still possible to write one that gets near to "DOS" kind of functionality, all by yourself. What you put in... you decide! It probably only runs on your hardware, or it doesn't make use of specialized features of hardwares, but you should ask yourself what kind of things you want to have for that particular OS and code away!

The tutorials here show you the start of writing one. If you get into the downloads area and look at one of those tutorials, you can even get a template OS to start from. Notice that Linux 0.01 is also part of the list and is a good read (and a chuckle!) to work from.

The first thing to do when writing an OS, I think, is "choose your tools". If you don't get them right from the start, you'll spend significant amounts of time rewriting it later. I am using gcc + nasm, since these are widely available and free. Then, start by building a hard disk image using the info from the site here.

You probably do not want to get started with a boot loader (it's really complicated!) and with the boot loaders available now, why should you? I did the following on Ubuntu Linux to create a hard drive image to start from. You go up to the point where they start to copy the guest OS, because that guest OS... is basically your real OS that will run inside qemu (more about qemu later).

http://wiki.xensource.com/xenwiki/InstallGuestImage

In the steps where "grub" gets installed... I had to manually copy a menu.lst into the "/boot/grub" folder. It didn't get created automatically. Maybe I should have "chroot-ed" instead into the "/mnt" and then try it again. If you take care of this image, you only will need to do it once.

Anyway, I copied in a menu.lst that looks like:

default 0
timeout 3
hiddenmenu

groot=(hd0,0)

title
root (hd0,0)
kernel /kernel.bin
boot

Of course you will need to replace the "kernel.bin" every time you recompile. I have this in my Makefile as such:

losetup -o 32256 /dev/loop1 ./test/hd.img
mount /dev/loop1 /mnt/temp
sudo cp kernel.bin /mnt/temp/kernel.bin
sudo umount /mnt/temp
losetup -d /dev/loop1
losetup -d /dev/loop0


And then you can test the OS through Qemu (better than VMWare, since it has a shorter time for startup and it is free):

qemu -hda ./test/hd.img -boot c -std-vga -localtime

Qemu is installable through apt, but the kernel module needs to be downloaded from the site. No worries... plenty of documentation there. On my Ubuntu Edgy I downloaded, untarred, ./configure-d and used make; make install and it all worked.

The cool thing nowadays is that you can downloaded various OS's from here and there and merge the concepts and see what happens. Also, an OS is not necessarily multi-user or multi-tasking, nor does it necessarily need to page out memory to disk and back again (you can simply page-fault and panic).

Unless you have a specific need or idea to follow, I wouldn't even start to believe it would be the start of the next "better" OS... But it's good to write one "OS" in your life to better understand how they all work.

Sunday, December 31, 2006

Happy new year!

Happy new year to all. The new year is starting off again with new promises, new resolutions, new business, new projects. I wish everybody well in the coming year and hope that they will get closer to the realization of their dreams.

There are some interesting developments on my own projects. Project Dune has launched a community site: http://pdune.sourceforge.net/ and I am waiting anxiously for people to start using it. The project has been rising in the popularity ranks at SourceForge and it's quite difficult to maintain momentum at these positions.

The good news is that I will be releasing a Beta release pretty soon. There is more working functionality, the main problems have been dealt with and so I am happy to upgrade the status at this time.

It probably won't answer immediately to your needs in your project, because the rules for ownership transfer and updating are rather generic and default. I'll only be able to make some good improvements when I learn more about how people want to use the software.

See you in the next year! I'm off to the beach to celebrate and will be back tomorrow....

G>

Tuesday, December 26, 2006

Measurement

I have read a book titled "Measuring and Managing performance in organizations". It really is an enlightener.

As many know, I am working on a project called "Project Dune" on SourceForge, which is related to quality. You could say that quality is highly dependent on measurement. And for measurement, you need to generate data and do that in such a way that you can historically compare one data-set with another. The end result is hopefully some insight in how you are doing as a team/department/organization.

Well, it is not really that easy. When you start to measure in an organization, you need to generate the data and both activities are coming at a cost of that same organization. Effort removed from normal production work. Well, obviously it is required to find a balance between going for production and generate the data anyway since you need to find a way forward to improve and without information backing up any decisions, you base all decisions on intuition, which in general can be very deceiving.

Project Dune is interesting in that the vision is that it should do the measuring for you. It is basically a similar tool like BugZilla, Mantis and so forth, but in addition to helping with standard administration and tracking, it also helps in day-to-day activities. And that is where the automation is plugged in, next to its envisioned integration points.

When you start to connect a system that knows about user activities, it can connect the data for its users and the larger the domain is where it is connecting "dots", the larger is the area that you can measure across.

The good thing about this is:
  • You get the measurements almost for free, always up-to-date and in real-time
  • You are better supported in day-to-day activities
  • You can spend more effort on your productive tasks without worrying about any process or dependent tasks that you do for others
Of course, it is not really that near to completion, but a BETA is coming out not too far from now. I'm just thinking about server-side call control (security) before I can even call it BETA. So far I have no feedback yet, but hope to see that happening at some time soon.

The statistics for the project are ranking up at the moment. The project is available at the third page in the rankings at this time (103) and seems to be going up still. A new ALPHA-3 release was just issued today. Let's see what happens next :)

Regards,

G>

Friday, December 15, 2006

Progress on Project Dune

I am progressing very quickly recently on the Dune opensource project.

It is a quality automation system. The objective is to make it work similar to BugZilla or Mantis in the sense that it maintains issues, but it also maintains information on:
  • tasks
  • customers
  • projects
  • functional requirement items
  • use cases
  • test cases
  • subversion commits
This way, the project is able to report across the whole area of development eventually. This also opens up the path to automation in traceability and probably therefore control.

I'm thinking of writing a theoretical paper on the subject of quality automation and the information that is supposedly lacking in quality plans, even though data is being "gathered".

check out the Project at Source Forge

Tuesday, December 12, 2006

Quality in Innovation

This post is my personal opinion and does not reflect the opinion of my employer... yadda, yadda...

I'm reading several books at the moment, one of which is the book to CMMi. This is basically deriving a vision for continuous process improvement in a company. But there are things seriously wrong with CMM. It is geared towards establishing a "static" state atmosphere, because if this were different, it would be very difficult to apply measurements across the process. A dynamic setting like an innovative powerhouse has severe problems adopting more rigid procedures due to the nature of their work. It is *not* a factory and they work with people that are intelligent and know what they are doing. Why insult them with procedures that tell them what to do, if they are the ones that know best?

The final level of the CMMi Zen approach is: "the optimizing level" :). I compare this to an engine room of a ship (I used to work in shipping).

This is where every process is defined and measured, you have all the spare parts you need, you have all the supplies you need and the ship hums along without a glitch. No leaking oil, no weird sounds, there are no bearings about to break or go. The chief engineer only walks around to tweak some settings on regulators for the form of it, puts his ear against a machine to hear its performance. It might tell the 2nd to replace a filter here or there or schedule an overhaul in the future, but overall the performance is so magnificent that everybody can go upstairs to the ship's bar. That's Zen, but software is pretty much different, especially when working on innovative and new products. Technology moves a *lot* faster in IT than it does in shipping. The speed at which new technology can be consumed on the ship is because they are users of that technology, not creators. And in software you may find yourself working on an embedded system to monitor cars first, subsequently followed by the latest technology in Ajax to create a project management system.

The later parts of my 729-page CMMi guide talks about "statistics" and "statistically managed subprocesses". Surely it must be evident that if you are not continuously repeating what you are doing, you are not building any historical data and thereby not creating the data necessary to improve your processes.

The real reason why it is so difficult to integrate with these quality visions is because in IT we are learning new things every day or encountering problems that are not standard. If I was working on a project with really new technology or bending the rules of what is possible with that technology, I would deserve a couple of days extra to find out ways and alternatives. I might choose a wrong alternative first and then have to rework. I might have to read through books of protocols in order to even start with architecture or coding. I may have people in the team that need to be trained.

Is a quality process that relies on repeatedness able to improve the quality of what I am doing?

....

I am not convinced!

( also check out http://www.joelonsoftware.com/articles/Craftsmanship.html )

Thursday, December 07, 2006

Software Quality

This is one of the first posts on software quality. Since 2002 I have been working on a project called "Dune", which is basically an automated system that helps you in the process of quality (it does not inspect, nor write code for you, nor writes documents), but if your company has strict processes regarding baselines, inspections, document control and especially traceability, this may be an interesting thing for you to try out.

The software is at: http://pdune.sourceforge.net and http://www.sf.net/projects/pdune.
Yep, it is GPL'd, so anybody can download, help out and use it, however you cannot take the sources and close it down for any commercial product.

There are some good rationales behind the project that I am going to write and design soon and put on the site. The whole work on the project is good to understand quality a lot better. I read through entire books about software quality (dry reading material about "process", "audit" etc. *yawn*), but in the end just discover that whatever they are talking about.... it can be automated! And how many tools do you know that automate traceability and have import functions for project plans, RF and UC documents?

A "quality process" in the end is nothing but the definition of responsibility. Setting the boundaries where the responsibility of one person ends and the other starts. So if you believe that a process is going to improve your software, I wouldn't really think so. The only thing able to do that are the people that you have in your time. The process should only help to create an environment wherein this team can flourish! And as written before, definition of processes is not always good, because if the process writer 'forgets' to write down a certain responsibility, it is likely that this issue falls through the cracks with no one to blame (blame it on the process, not yourself :).

The rationale for adopting a software quality standard doesn't really have much to do with actual software quality. That is, a software standard certification need not be in place in order to write good software. Good quality depends on good people, not on processes of any kind. It may help, but only if the process is defined and adopted by the people in the company. Rather, a process must be seen as the formalization of how work is done in that company with that culture, not an enforced method of work by a select few in the organization that read a good amount of books on software quality and then run around crazy with the theory.

I have not yet read CMMi in its entirety, but I understand what it is about. I am not yet sure whether what I am building will comply with CMMi or vice-versa. I do expect though that the "rules for certification" of CMMi fit in nicely.

Any quality "system", whether this is a quality plan, a set of software tools or adopted method of work should aim for a couple of objectives to achieve software quality:
  • transparency
  • traceability
  • control
Control is the means that a team has to exercise control over the 'states' in the project. There are initiatives, purchases, work items, bugs, features and change requests to be managed. The better the software is aligned with how the team wants to manage this information, the more effective it will be.

Traceability has much to do with the CMMi and ISO standards. This is often interpreted as a "paper trail", but I argue that it might just as well be a set of records in a database (it is actually much better). Traceability has much to do with "auditing" as well. To know when, how and who is basically the question that for each initiative, bug and feature needs to be answered.

Transparency means a system where the information is close to the surface. One should not have to open a separate locker with a key that only the manager has to go through drawers of paperwork to find out what happened to bug #35. This is one extreme transparency problem, but it shows how transparency is important in a system. Transparency is basically a measure how easy it is to get to the information in the system. The easier it is, the more value the system generates for its users.

So, this is what I am building. An automated quality support system. The system by itself is incapable of guaranteeing quality, but it helps to lift the effort in maintaining the audit chain and ties control, transparency and traceability together. This is very important for people involved in software development. It relieves the people involved from a couple of hours of effort a week (as an estimate), which can be directed to something really productive.

Tuesday, November 28, 2006

Measuring Performance

Here is a very interesting link to measuring performance and its results:

http://www.joelonsoftware.com/news/20020715.html
It is already common knowledge that whatever metrics (goals) you put in place, people will work towards maximizing that particular goal. Having several goals initially creates a lot of confusion and discussion about the weight of each, but eventually this settles and creates a certain "culture", which you could consider as a weighting of the objectives of that group of people.

A proper means of measurement has everything to do with renumeration models:

A renumeration model for distributed development

Setting a single and individual goal for productivity will fail. People will refuse to work together, there is absolutely no cohesion in the team, only competition. Beyond this not being a 'fun' place to work, it is very stressful and breeds backstabbers. So we need a complementary force to pull the team together. Some competition is good (as it 'advances', but it cannot outperform the need to collaborate).

( There is also some more information on Joel's blog about incentives. )

In the future I believe that the "an-hour-a-dollar" system of renumeration will eventually wane away. Simple because more people will (want to) work at home, collaboration becomes possible with enormous installations of fibre around the world and there is no sense creating a huge, expensive office infrastructure when you spend only a fraction of that cost to develop one's home office.

Fiber installation in Iceland
The traditional hour-measurement systems, if installed at home, can easily be circumvented and cheated. So there is a need for a set of personal and group objectives with appropriate weights, tied together in a renumeration system. Maybe group objectives are better explained as "individual penalties".

You need to reward people that pull the team forward (individual objective), but you cannot reward people that leave the worst work to others (group objective). You cannot penalize a group when one worker leaves much to be desired (group objective), but you cannot penalize newcomers straight from university from not performing too well (individual objective).

In my previous post I discussed the possibility to tie company profit into this system of renumeration (making company profit have an effect on 'salary')

The reasons?

"Having a job" is the most wonderful thing in the world. There is no stress involved if you compare this to actually owning a company. Maybe the knowledge workers of today, consistently aiming at more benefits should feel obliged to share at least some of the burden of finding a proper path for your company to grow. Either through a negative incentive or through a positive renumeration?

The other thing is that it puts more pressures on companies to leave work behind that is simply not productive. You'll be amazed how much is still being done nowadays that does not contribute to a final product. In theory, such a tie with profit would make the company much more flexible and the human capital organizes itself better. But I guess only when it decreases salary. There are some schemes for profit sharing, but this is on top of the guaranteed salary, so the incentive to aim for it is much lower.

Monday, November 27, 2006

Smart homes in Korea

This article talks about smart homes in Korea:

http://news.bbc.co.uk/2/hi/programmes/click_online/6179868.stm
I did a project on home automation some time ago and we're just scratching the surface. There are quite some people out there who see HA as a hobby, but the market is starting to get itself organized on better features.

As soon as the more mature electronics producers will start to develop and market some better gadgets and there are standards for ex-change of data, this may actually set off on a very fast pace.

In the vision of HA, you need to see the house as an extension of yourself. It does not only "manage itself", it also manages things around you and all of the house's inhabitants. As soon as the house is empty, it turns itself off :).

The computer that you are now in front of can actually become an integral part of your house. There is not necessarily a need for this computer to stand in one place. Why not move around, do your stuff and still interact with anyone you are talking no matter the room you are in?

The advances and implementations of social networking show how we are picking back up the ways of interaction and finally start to expect our machines to serve us, instead of being slaves to the keyboard or "program implementation" (that's just the way it works, now learn it!)

Korea's smart home shows it is not just about ambience, it is also about providing feedback on certain stuff like washing clothes... Being "smarter" than the inhabitants and giving them tips before the bad things happen.

Is the actual step in HA a house that takes care of its inhabitants? Can the electronic house of the future act as a digital baby-sitter?

Sunday, November 26, 2006

Royalties....

100. Musical instrument shops must pay an annual royalty to cover shoppers who perform a recognisable riff before they buy, thereby making a "public performance".

http://news.bbc.co.uk/2/hi/uk_news/magazine/4566526.stm

Monday, November 20, 2006

Australian copyright legislation rushes on!

A post about copyright law in Australia this time. Australia is seeing new legislation rushing through Parliament that would incriminate many regular Australians now and in the future by copyright violation.

New Copyright Laws Risk Criminalising Everyday Australians

Happy Birthday Song
It's very silly, but even scarier. At the moment, luckily enough, everybody I talked to finds this appauling and very silly. But what if this trend continues? Then you won't be able to whistle or sing anything from the radio in the street anymore since that counts as singing. The band next-door won't be able to perform in the bar two blocks away unless they pay royalties...

The danger of individual freedom and liberty is obvious. But what of the homogenisation this may inflict on our culture:

Consider a futuristic scenario where a very large computer will analyze written pieces of music with pattern-matching algorithms to see how much of the song is based on other songs and spews out a number which is what the person should pay in royalties?

If a system exists where we have perfect control over who listens to what and how this is distributed, then we will only be able to hear some new piece of music when you pay for it. Consider the bad implications. The radio stations and TV are the only means that you can hear something new, which most likely leads to massive homogenisation of the music industry. It is no longer the consumer who controls what is cool, but the production companies (maybe this is already commercially determined in this way at the moment, but not yet technically enforceable).

As a human being there are only x hours in the day / month that you can be provided entertainment. This means we continuously make changes on what to hear or see. If there would be zero piracy, the divulgence of unknown bands is very low (since there is a cost associated with its purchase and people in general don't like buying something that is not known what it is beforehand).

A little bit of piracy is just plain healthy, especially for culture. Music and films are not "material goods" where it can be directly associated with a cost. It's a lot more or less than that.

Check out the process for making a film:

Peter Jackson will not be involved in "The Hobbit"

Time for a new business model!

Monday, November 13, 2006

Telecom Billing Systems

I am reading on the main book that is refreshing my knowledge on billing systems. This book was bought on Amazon and some guy has written in quite good terms and vocabulary what a billing system really is all about without all the technical fuzz and buzz.

It is describing customer management, provisioning from other systems, the billing interfaces towards finance and fraud detection, the focus on the account as a billable entity, reconciliation, usage calculation, product charge calculation and so forth.

Well, it is going to take me some time to finish up, after which I have another book ready that is overdue (The World Is Flat) and another book that I have read from A - Z but have not sunk too much into the exercises which is an introduction to accounting.

Another thing I am doing at the moment is setting up a new experimental site that shows a menu for open source projects. Not sure where this leads at this point, could be anywhere, but it is a nice initiative to finish. That's right, I sort of started by mentioning it to someone but never got to the point of actually starting due to a lack of web server space, but I remembered that Google offered pages as well.

It's at http://gtoonstra.googlepages.com.

Sunday, November 12, 2006

Human (Parallel) Processing Power

I was surprised recently about the power of parallel processing when many people focus on the same objective. Some person had the cellphone lost or stolen and some hours later her Flickr account showed a photo taken with the cellphone of two rather ugly individuals. The background showed only part of a wall, a window, a reflection and part of a lantern and a wire.

She posted this picture on some well-read forum and within minutes the readers resolved the actual location and resolved other questions about the identity of the individuals in the picture.

This shows me that certain small tasks with a desired result (identification, problem resolution, etc.) can become very easy as long as you have a large enough audience to process it. There was a large amount of noise generated by this audience, but inbetween some very interesting resolutions and suggestions.

There are probably certain limitations on the efficiency of such an effort:
  1. The end result must be clear and objective. This rules out open questions. (how to....) is not a good question to put in a processing task.
  2. It must be a small enough task that only requires knowledge and creativity to solve. Actual manual effort would be very difficult to organize and coordinate.
  3. It must be a problem that lies within the expertise and experience of the audience.
  4. The community should be able to share references to documented knowledge or images.
Considering what this means for a larger company... If you have say 600 employees and all 600 of them put their brains to work for five minutes on a single question that has a clear objective, then you get the collective experiences, expertise and insight of all those 600 people plus an aggregate of 50 hours of worktime at relatively no cost at all, since 5 minutes loss of work is not very significant in a day. If the problem is relatively within the expertise area of this audience, the resolution path might put you right on course for a perfect resolution.

I can imagine that for the single person to develop a good resolution path to this problem it would take much longer than 50 hours and it might not be optimal. There is also no knowledge sharing (since the result is published otherwise).

Is it better if the result is developed and shared automatically by all people involved? Can the audience that is participating steer itself in the best direction for the resolution and auto-correct eventual mistakes?

I found some interesting sites on this topic that also touch on something else, human-computer interfacing on the brain-level:

Yasir's Blog
Integrating computers with human brains
Riya
Now, on another note, already scientists have developed software and hardware that allows a paraplegic to move a cursor on a screen. There are two things to consider here. With bionics we can augment human functions and allow the human to improve, but vice-versa we can also create human interfaces that would complement a computer so that some automated process becomes much easier.

In a way, Flickr is something like this. It receives keywords from people browsing the photo's, so-called tags and then indexes photo's accordingly. The classification of things thus happens with people whereas the rest is simpler to automate by machines.

Maybe we should run some experiments by thinking of some not too difficult objectives to achieve and see how we perform in the resolution of that problem.

G>

Tuesday, November 07, 2006

Flourishing in the private sector

I'm reading up in a book from "C.K. Prahalad", The Fortune at the Bottom of the Pyramid. This book is mostly looking at a portion of society that is not totally impoverished (extreme poverty), but certain means at its disposal to make purchasing choices and maybe even has the capacity to save (capacity being not the same as possibility). Well, if it was meant to address the totally impoverished, see previous post that explains how I disagree :).

The questions that are being raised now for Brazil are:
  1. Does India / China relatively have many more startups than Brazil?
  2. Is the global and international activity (objective) of those developing countries more aggressive than the Brazilian activity?
  3. Are the legal systems for China and India better developed (maybe in definition of law, otherwise in its enforcement) than Brazil?
  4. What are other important factors to take into consideration when considering stagnant economic activity?
I'm making assumptions here that the production capacity of Brazil is much higher than current production. The book of Prahalad has a chapter on corruption that is for me a very interesting read that goes further than just eliminating corruption, it also offers solutions. Part of it is digitizing the local or state government and reversing the role of government by not making citizens comply with the government rules (which has never been seen as the objective of government), but to make the government adjust to the wishes of the citizens.

This concept is also known as e-governance. The introduction of this will surely produce an initial surge in corruption and malfunction, but later on it should produce a better balance of power and consistent treatment and application of law, if we assume the knowledge of the general citizen sufficient to be part of this system (that is, part of its reasoning must be based on individual interest, but a grand part in the interest of a group, which is society).

India is doing this in one state already, the state of Andhra Pradesh. Here the political leadership wanted to turn around the excessive burden of corruption especially in the registration of land. They invented a computerized system that makes it much harder for corruption to manifest itself. The result was that the public no longer needs to pay the corruption fees and therefore also change their perception about the level of corruption in government.

Since then, the state has sought to automate and digitize many more public services. In this way, the citizen also becomes an agent in the system and it's more of a "self-help" way to interact with government. Government, after all, is a representative of the people and serves to administer the access to public services by those people. It has power to make laws, execute laws and judge people that break the social contract with the government.

One side of politics argues that a country that lacks an effective justice system has little chance to increment economic growth because there is lack of confidence in people that property rights are maintained. This reasoning goes all the way back to Hernando de Soto. (the other political side mainly talks about social injustice and social inequality, causing problems to generate more wealth in the long run).

I tend to believe that when contracts cannot reasonably expected to be enforced, it becomes difficult for new entrepreneurs to engage in business because the non-compliance is seen as a threat. The current legal system in Brazil seems to reflect this. Too many people and legal professionals I met are faced with 4-5 years to solve a particular case. Only when this can be expedited or alternative measures for contract enforcement are found will we probably see some higher growth rates. (China, for example, has enormous corruption and a poor legal system, but the contracts are enforced between the business people themselves, more by honour than by a legal framework that backs them up).

Final word about e-Governance... not only is corruption reduced but in the example of the Indian state... The workers there were required to stand in line 4.5 hours per month to pay their electricity bill. Prahalad estimated this to be U.S. $45 million in one city per year. So, by improving the access to information and access to public services by automating and digitizing those, we can reduce the level of spending (less gov't employees and wages) and the workers do not have to spend 4.5 hours per month standing inline and continue to work.

That sounds like a win-win situation...

Monday, November 06, 2006

The End Of Poverty?


I've just finished the last bits of this book "The end of Poverty" from Jeffrey Sachs. It is an amazing read and it shows the hands-on experience of this professor, now Director of the Earth Institute.

The book explains how poorly understood the concept of poverty really is and even sometimes how much poorer are the understandings of the solutions to counter it. It talks mostly about government interventions as a necessary solution to combat extreme poverty, which is defined by the World Bank as those people living at less than US$1.08 per day (Purchasing Power Parity-adjusted).

I found a couple of arguments and statements very intriguing in this book. Also I had a look on wikipedia which I consider a source of general public opinion to compare the views of the book with what "the world believes".

If you read the text from the link: Wikipedia Cycle of Poverty then you will notice that there is a lot of mumble about education, privatization and so-called "free-market solutions". The IMF and World Bank are not just subscribing policies to poor countries on what to do with these elements to alleviate poverty, but also maintain in many cases a political agenda.

As the book argues, the "extreme poverty" question has not much to do either with education, privatization, nor so much government corruption. It is on a lower, more basic level than that which is that the people do not find the ability in their country to save and therefore invest into a better future. This happens in Africa mostly due to a number of issues:
  • 90% of malaria deaths occur in Africa (which can be treated, but the African mosquito really likes to bite humans rather than cattle, plus it is warmer, which intensifies the impact of malaria 3-4 fold).
  • AIDS literally rages through Africa in Sub-saharan countries.
  • The grounds on which they attempt to grow their crops are totally depleted of nutrients.
  • Lack of infrastructure
  • External trade-barriers
For some more background on the above, check out Wikipedia again: Poverty diseases, Global Distribution of Poverty, Child Malnutrition (measure for hunger)

There is actually an interview with Mr. Sachs about his book over here. It explains better the reasoning that the book is trying to make clear.

Many people naturally assume that poverty is 100% about corrupt governments, an idea that if governments and public institutions work perfectly, wealth generation happens naturally and economic growth results. The book has shown me that this is not necessarily the case.

In some communities in Africa, the people need all the food they grow for themselves, so lack any savings / investment basis to allow them to grow further. Plus that the impact of diseases, lack of education for women and population growth every time undercuts any possible growth that could have occurred.

Since there is generally quite a lot of inflation in these countries and the government coffers never fill up, but instead service foreign debt, the country just turns poorer and poorer, without the ability even for the country to invest in national infrastructure. One African country's funding (relief) was US$100 million against the servicing of debt of US$600 million. So more money flows out than comes in. The actual need for foreign aid for that country stood at US$1.5 billion.

Figures also show that even in countries where corruption is abundant (corruption perception rank standard), there is still consistent economic growth. Countries like Bangladesh and Indonesia for example suffer from enormous corruption levels, but still maintain 2.0-3.5% economic growth. India is growing at 3.5% and this has about half of the corruption in those other countries. (source: Global Corruption Report 2006, look for "Table 11.1: Corruption Perceptions Index 2005")

This is not to say corruption is not a factor, but it is not a factor as strong as we commonly tend to believe. The key factor in reducing corruption is education, which is an instrument to create better jobs, more understanding and more people power to fight those corrupt powers (which in countries like Africa is unthinkable, since corruption is basically a system that can only be broken by a general level of trust and equality between the people). So, in this reasoning, poverty and corruption have a very strong impact on one another.

The statement of Sachs is very interesting. Whereas previously there is a silly debate on solving the wrong problems (that do not alleviate extreme poverty, but have marginal impact on economic growth and requires a certain maturity), there is now a shift towards a more understood debate on poverty. That every country has its own specific structural problems that lie at the root of poverty (against the convential and conservative wisdom of Thatcher and Reagan days), and that a program can be built for each country that helps to get it back on track. Still, this way, you cannot expect it to perform the same as another country, it will take literally years, 50 years at least in many cases, but at the moment the African countries are declining in health, education, economy and this gradual decline only worsens the situation more, which causes more and faster decline.

The level of expenditure that is currently given to alleviate poverty in Africa is not enough and needs to expand. The US has yearly spent $15 billion on extreme poverty relief, but US$450 billion on the war in Iraq for example, where the destructive effects of war only cause more poverty and hatred across the world.

This is looking at the problem from a macro-economic and governmental level. Are there innovative opportunities related to this problem? To monitor and allow the flow of relief funds the richer countries need 'peace of mind' that the funds are applied properly and do not disappear along the way, neither what is bought with those funds. Solving that problem is worth millions... Or what of the agricultural technology opportunities to enrichen the soil for better crop planting? Using available technologies to allow communications to take place for a better-working market (where there is agricultural trade). Off-grid generators for providing electricity in housing (for late-night studying) that maybe could power a single low-cost computers or DVD-player (for health education for women, birth control, counter diseases at the community level, etc.)