Charmed by pythons

In my latest project I'm using Python to construct the basis for a GUI application. Because one of the main design goals is to make this as modular as possible, it is also used to construct an application messaging bus and another object to keep application data in one single place. My experiences so far are extremely positive. I'm used to formally specified languages where there is no possibility to become confused about the meaning of a parameter or what its type is. This makes it slightly easier at times to understand what a parameter is doing, but at the same time removes some of the flexibilities that these programming languages offer. Python seems to be the ultimate mix between form and function, although it takes some time to get used to the idiosyncrasies of this particular language. Once you stop worrying that your application isn't going to be used after three years anyway and that nobody wants to extend your particular piece of code, Python becomes something that you can start to embrace.

What I had to get used to at first:

• How python expects you to indent your code. I set my editors to 4 spaces instead of tabs to make my life easier. Still, you download a snippet of code from the Internet and you end up rewriting tabs as spaces and vice versa.
  
• The same indentation levels is how scope is managed, whereas C and Java use scope braces.
• The ability to simply assign a variable some value and how it persists over time. There is still a gotcha or something to remember here, because sometimes variable assignments are persisted in the instance and not the class. But usually this turns up soon enough.
• Some short-hand notations for iterators over collections, sets, lists, deques and dictionaries. It takes some time to get used to how double braces differ from brackets and from square brackets (they mean different things), but when you know Java and the differences between sets, maps and lists, these notations become rather natural.
• How some declarations or references of C libraries eventually must be interpreted to understand which classes must be instantiated and where C-enumerated types are declared in the python bindings (at least it's consistent!)
  

The awesome thing in python is that it's not just something you run on the command line anymore. We're using this together with the Gtk 2/3 libraries, Clutter and libchamplain. These are highly graphical applications written in C or C++ and python with the GObject bindings give you access to all the functionalities in those classes.

The two coolest things in python is that we now have access to a very clean and empty user interface application that we can enrich using a set of plugins. If you know what a model/view/control (MVC) separation of concerns is, then python definitely knows how to support that. For our data and for our messaging bus, we've created a singleton object in the VM which every object can get to in a very simple way. Any plugin can declare data items that it wants to store and it can itself use the message bus to declare new kinds of signals that other plugins can react to, or it uses the messaging bus to declare interest in messages of other plugins.

This way, the application is 100% modular, but there's still a sense of control onto what kind of data is stored, where it is stored and it warns developers when a plugin wants to get access to data that hasn't been put there in the first place.

The plugins we've defined are all of a specific type and have their specific pre-determined uses. Communication plugins usually run in a separate thread and they're responsible for opening their own sockets. They then receive or send information from/to the system. Using the messaging bus notification signals, they extract information from the model and send this on, or they receive new information from the environment and add this to the model.

At some point though, one needs to be aware that any application can ever do so much. The multi-threadedness is highly governed by the ability of the main thread to keep up with whatever is going on in the environment. That is... in a graphical environment like clutter or gtk you can't update or manage components from just any thread, but you can only do that from the main thread that is running Clutter.main() or Gtk.main(). This usually means you add notifications to the message queue of the main thread, which is only emptied when the main thread becomes idle.

Thus... if you are in an environment where lots of user interactions happen and the UI is never truly idle, the communication message handling may start to delay by quite a bit and you may notice 'halts' in the UI updates from these systems. Because of that, this is not necessarily the way to go for everyone. But this is the best of both worlds really... you can't have blocking sockets in UI thread code, you can't/shouldn't obstruct the general UI thread with system messages (making user interaction choppy) and other considerations like that.

So, the main design concepts of this system are:

• Keep data in one place wherever possible (if multiple plugins use that data, don't copy it for every plugin).
• Allow data to be private to plugins when no other plugin or code uses it.
• Pass in required references to objects that make sense to be externally referenced. Because the use of each plugin is clear, you can also separate these.
• Communication plugins probably need a separate thread for communication handling. Be careful with blocking sockets, because UDP sockets may continue blocking forever. Therefore, TCP sockets may be blocking (as far as you shutdown and close them). UDP sockets should not be blocking.
• Use a special singleton instance for a messaging bus where messages are declared on and where hooks can be inserted. This allows you to manage mbus code in one place and you have a nice intermediate class that passes signals around.
• Do not pass large amounts of data on this message bus. If large, hierarchical pieces of data are manipulated, store them in one place in a model and allow plugins to query them if they are so interested on the receipt of these signals.
• Define what your UI should look like. That is probably the only thing that ends up being highly specific code to the application. But if you have your mbus+model objects defined already (and these are generic), you'll find the main application window is nothing but a 'shell' from which plugin code is run and the logic is defined by what plugins do and which kind of clutter/gtk classes are contained in your widgets.

So yes... I've been slightly charmed by the elegance of python in certain expressions. It's a rather mathematical way of seeing things, but it starts to make sense a lot. The abundance of libraries, extensions and base libraries, most especially its support for Gnome repository bindings for all sorts of purposes make this a very attractive language to program in.