Monday, January 30, 2006

Back from WSF 2006

I'm just back from WSF 2006. There was some very little coverage on CNN about it, the majority of CNN was about the World Economic Forum in Davos.

As said, there was a lot of Bush-bashing. However, this was not a gathering of just 'leftists' etc., there were many different people there. It is a bit sad to see that HC gets all coverage on CNN with some radical statements, but the actual topics being discussed are not really being published. Also, although there are some sessions highly politically motivated, the actual essence of the WSF is social action. This does not have anything to do with politics, just changing governmental action whatever the government is composed of.

I went to a couple of interesting meetings:

* A workshop on the legislation of labour in developing countries

* A presentation about the work of a foundation that protects low-skilled labours in the Amazon, so that these people do not turn into slave labourers. The definition of slavery here is not specifically "being traded", but it means co-ercing people into labour with very bad conditions by making it appear as if they owe the owner of a farm. This is done by not declaring the bar bill, charging for the trip from their home-city to the Amazon (the 'gato', which is the guy who lures these guys, is posing as a travel agent and states you have to pay to get there) and more of these practices. For these reasons, one month later they get one bag of rice, maybe some trinkets, and that's it. Some drink water from a hole in the ground. Housing is basically a rooflet of wood with some poles underneath. Beds are wooden without mattress. When working with raw coal, no protection is given against pore-infiltration of coaldust... etc... etc... The federal government is on it now, clearing out the farm owners that violate this solely for the purpose of profit.

* A presentation about incubating "cooperatives" of low-skilled labourers in Brasil. This is where recycling-goods collectors can collaborate together, get better rights together and learn about administration and do things together basically for the good of them all. New hints, new ways of work, etc...

* A presentation from the "Mitterand Foundation" about the new society, which is suffering from "predatory capitalism" and especially why this is the case. They explained issues like water management, the right of people to have at least 4 liters of water a day, the privatisation of drinking water! , terminator seeds discussion (seeds that can grow only once and require farmers to continuously buy the seeds from the same supplier).

All in all, a variety of topics. It's definitely worth a visit and next year in Kenya, I am already planning to go again.

Sunday, January 15, 2006

Keep It Simple, Stupid!

I've worked all weekend on the Patty analyzer and things are getting quite stable. It has been ported to Linux and Solaris in the meantime. It should now be in a state where people can actually start using it more thoroughly and depend on it.

Download: Patty

Other news is that things at work are also going smoothly. We've recently had a long port of entity beans to Hibernate and also an effort to move from an old JDK1.3.1 to JDK1.5.0. One of the most ugly hacks I've ever seen in that project were that Java code compiled for JDK1.3.1 was actually stored in the database and loaded through a custom classloader. *ugh*.

The title of this post refers to that... Just because you can store blobs and code in the database does not mean that you should. An "innovative" idea may be really cool to code in, but it certainly does not mean that it is a good solution. Far from it. And having 4 different executables with different names that all do something similar is really annoying too. Like the Oracle client installation or ClearCase.

Or projects with dependencies on large frameworks of 80MB in download size. Some of them having licenses or evaluation codes that you need to register for.

My objective with Patty is to make it a very simple thing to install and run. Download Tomcat 5.5 (around 7 MB in size), unzip and run it. Deploy 'drill.war' in there. Set up the agent library (42 - 160kB) in 5 minutes in a Java runscript and people should have a profiler ready for the project.

Thursday, January 12, 2006

Progress on Patty

I've been working hard to push out an alpha 0.1 release. This is a sneak peek into what this is evolving into.

The actual GUI doesn't look particularly pretty, but the functionality is all there.

At the moment, I'm finishing up on heap analysis. The first thing I did was the ability to iterate the whole heap to analyze heap usage. Immediately thereafter I worked on the ability to trace objects that you can reach from all instances of another type of object. For example, if you are analyzing a web server application, you really want to find out what kind of objects are referenced in your HttpSession and the total size that is referenced in the instances overall. My idea is to average this out later, so you get an understanding of the average session size.

Garbage collection needs some more work for visualization.

To be able to get to alpha-1.0, I'm still verifying with runs in JBoss now. That seems to start up quite nicely at the moment, so I'll probably look at another J2EE application for download somewhere, deploy it and actually start using the thing for real now, instead of the meager test application.

Sunday, January 08, 2006

Patty went Alpha

Patty, the project for Java Performance Analysis, has gone to version Alpha today.

I've included a selection mechanism for code coverage and method execution analysis and thread contention analysis is working too. It can be downloaded from sourceforge.

I'm currently working on the Beta version, which needs to include dynamic class instrumentation or de-instrumentation. This has just been tested and is working fine now. There is just a little bit of work on the command sockets that I implemented, basically a TCP stream that communicates between the web-application and the agent library.

Another feature planned for Beta is garbage collection analysis and 'which-object-references-what'. The idea is to show a couple of 'root' classes (objects in the VM, but will be shown as classes in the analyzer), so that you can easily track where memory is being held or where it increases.

Maybe a couple of those analysis dumps can be used in the future to perform some heuristics checking on memory leaks or whatever.

After Patty goes Beta, I want to write up some stuff on performance analysis using Patty, so that the tool can be used much more effectively. Some charting is going to be used as well and some better heuristics on the averaging and trouble points ( potential 'trouble' page for instance, where the problem points are listed).

Thursday, January 05, 2006

Garbage collection in Java

I'm going through a lot of information lately about garbage collection, starting from the 1.3.1 JVM. Interesting process overall. The current project I am working in uses the 1.3.1 and is going to upgrade soon to a newer JVM.

The collection itself does not truly impact any server application until the loads get higher and the space settings suddenly start becoming very inefficient. Especially in the 1.3.1 VM and with a relatively very large memory heap maximum, the full gc can take up to 25 seconds to complete.

This is mostly due to inefficient use of Java objects and holding on to references. This happens when server applications decide to "cache" a lot of information to prevent database queries. If the sizing at the design stage is not properly assessed, then the application will perform fine for some 200 records, but if this increases to 20,000 then certainly someone needs to take another look.

My advice is to keep caching to a minimum or use a standard caching framework like 'ehcache' to do this for you instead. It is certainly better than making your own implementations in your application, plus that the caches there work together with Hibernate and exist closer to the persistence layer.

Java 1.4 is already improving this by parallelizing garbage collection. This has enormous positive impact on the server process, but only when the server really gets busy on CPU of course. I'm going to read up soon on 1.5 collection and hope to post some useful documents in the next 6 months to be posted on the Patty site, along with the performance tool that I am working on.

Regarding the performance analysis tool, it is coming along nicely. Already there is a framework for web access now that is going to be expanded and the agent library seems stable. I'll need to expand the testing application to also have some thread contention and to load more classes, libraries, maybe even include Derby + Hibernate and CGLib so that I can see how it performs with proxies and dynamic bytecode instrumentation.

Until then!