Well, back from PyCon. It felt very busy... whether it was a BoF, or working on my presentation, or working on FormEncode so that I wasn't talking about vaporware... anyway, it was very busy. But very good.
Anyway, if I didn't have time or the leftover thoughtfulness to blog about it at the time, I'll try to write about some of the ideas after the fact.
First up, a couple of the lightning talks I gave:
If you reload modules when changes are detected, old references persist... When any file has changed, restart the system
We have to restart the system to fix code - that's not very dynamic!
For long-running processes we need to hot-fix code and resume (not restart) - Erlang and Smalltalk and...# Isaac Gouy
# Isaac Gouy
Lots of Python systems have tried hot-fix and resume. They don't work very well, though some may get close. Some of the problems are actually quite challenging regardless of environment -- you have to take each object and figure out a way to update it according to new specifications. So you either have to specifically specify how to upgrade each object (which is burdensome and is more code that can contain bugs, especially burdensome when you are just doing active development and have no valuable objects).
In Smalltalk, sure, you can add another instance variable -- but they all end up as nil. That very well may be an impossible situation except when updating an old version of the instance variable. So how do you deal with that? Allow for the possibility of nil, even though that's a transient possibility? I'm sure there's tools or pattersn to deal with this, but it can't just be automatic in the general sense.# Ian Bicking
CLOS - the object system of Common Lisp - solved these update problems quite early on. You have many ways to update instances on the fly when classes change. Sure, it's not fully automatic - it can't be fully automatic. But it is very easy to do. Erlang goes even further. Of course that's because of it's primary use that _needs_ hotfixing.
>how do you deal with that?
Pause the processes that use the objects we wish to mutate, mutate the objects, initialize the changed fields, resume the processes. (The processes never see the "impossible situation".)
>specify how to upgrade each object
>(which is burdensome
Burdensome?! Compared to taking down a live server and throwing away a long-running computation?# Isaac Gouy
Well, it's burdensome when you're only working on a development server, and/or you aren't performing any long-running computations. (Note that, in Webware, restarting the server doesn't cause any lost connections.) In a web environment, long-running computations aren't used that often, or if they are you might already have mechanisms to keep them running outside of the main server process. But yes, you have to consider how you'll restart the server safely.
I can imagine it would be more of an issue if you were using objects that were persisted for a long time, and you wanted to upgrade those objects and maintain a list of "patches" that you could use to upgrade other objects when you deploy the software (though that's a challenging problem in itself). That's not generally a style that's used a lot in Webware, and probably not the style used anywhere where you don't have an object database.
And if you have an object database... well, then you really do need to upgrade objects, and with considerable robustness and with a formalized process. But that quickly becomes a quite different discussion.# Ian Bicking
"The Problem...
In a long-running process, old modules stay in memory after their source file has been updated."
If it's acceptable to simply stop/start the process, then it's strange to describe it as a long-running process.# Isaac Gouy
Well, "long-running processes" makes sense in the domain that I am most familiar with -- daemons and servers -- which can usually be easily restarted, but are not restarted for every request (in contrast to CGI or inetd servers). But yes, you're right, it is confusing terminology, since it means different things depending on context.# Ian Bicking
"If it's acceptable to simply stop/start the process, then it's strange to describe it as a long-running process."
Somehow, all this reminds me of that famous "Worse is Better" paper. ;-)# Paul Boddie