I'm working with the HTTP protocol everyday. It was initially developed as part of an effort to serve static HTML files that have no interactive content or relationship with the server. This was also in the era when servers were not as powerful as today, let alone as cheap.
From a hardware and traffic perspective, it makes sense to terminate a connection once the files have been served. Keeping a connection open for every user means consuming resources on the server. It always includes keeping a socket open. It is required to re-work slightly the concept of a thread serving the user tightly integrated with the socket.
The bad thing about terminating connections is that one needs to communicate a session ID with the client and that "execution context" cannot be maintained and needs to be re-loaded every time. Every new request needs to regain full sense of a session and potentially reload a lot of data for that particular user. Having a separate context of execution per user that remains active (without being serialized) can help in making the application more useful to others and more reliable ( back button? ), because every new action is still related to the same context. (Context management in HTTP applications can be a problem, because you don't know whether someone will contact you again and you do not necessarily want to remain with a large hierarchy of objects for a user if it is not necessary and the user has gone away. This is why session timeout was introduced).
Probably servers should not deal so much with interface code (HTML) as at the moment is the case. The application revolves around re-creating the full GUI at each new request. AJAX has reduced this to only parts of the user interface.
An approach with pages and XML injection has already been proposed before by Microsoft, which was further refined into "AJAX" by Google. Maybe this can be taken a step further to create a browser that downloads a template pack or templates 'on-demand' where the actual data that needs to be injected is inserted from a separate call. I'm not talking Ajax here, but more something like "continuous RMI" where the connection with a container is never terminated. This enables the web server to become quicker in serving requests for smaller communities, because it does not need to reload the context, set up or deal with security checks done every request, etc.
Server push already become broken due to NAT firewalls, so I don't think the solution is there...
To be continued...
New tool in town: KnowledgeGenes.com
7 years ago