From: Michael Hudson Date: Fri Dec 5, 2003 13:51:40 Europe/London To: Bill Soudan Subject: Re: PEP 310/object lifetimes? Whew, this is getting a bit long and a bit deep! Hope it makes some sense... Bill Soudan writes: > On Thu, 4 Dec 2003, Michael Hudson wrote: > >> More or less what I guessed. Which ICQ client do you use now? I've >> settled on centericq for the time being. > > I don't use IM in general much anymore. I find it too distracting and > too > much of a temptation if I'm trying to get work done - I'm terrible at > multitasking! When I use one (at home sometimes) it's AOL IM, since > everyone in my social circle has an account there. Fair enough. > Usually Gaim, but sometimes naim if all I have is a text console. > > Though I think I heard somewhere ICQ/IM are completely interoperable > now? News to me, but it wouldn't surprise me given who runs it all... >> [Locker class] >> >> Well, false is perhaps the wrong way of putting it... contrived, >> maybe. > > Interesting the different perspectives :) Yes :-) >> I guess you could say that part of the art of programming language >> design is deciding which notions to conflate in this way and which >> not. > > Right, I agree. I suspect we think about the whole RIAA concept from RAII, I think :-) > different angles, probably because of different programming > backgrounds? > I try to explain mine below. > >> You can make a case for the conflation of holding resources and >> lifetimes of objects, but I'm not sure I buy it. > > I've never honestly thought about it much, just used it, but I've > taken a step back and tried to analyze it after your last email. It is interesting to think about these things, isn't it? Even if all you do is understand your own position a little better... > I still find it very natural if you consider the object as a gateway > to a resource, think file handle in C or file object in Python. You > only have access to the resource when the object exists. Destroying > the object does not destroy the resource, merely, your ability to > access it. I think a key point here may be your using the word "destroy"... > The method you propose in PEP 310 introduces an additional layer from > my > perspective: > > 1) __init__: create resource access object > 2) __enter__: 'enter' resource access object (acquires access key) > 3) ... > 4) __exit__: 'leave' resource access object (releases access key) > 5) __del__: destroy resource access object Note that __del__ methods do not destroy objects -- they are called when objects are about to be destroyed. > vs. > > 1) __init__: create resource access object (acquires access key) > 2) .. > 3) __del__: destroy resource access object (releases access key) > > What value does the extra layer add? To me, it's unnecessary > complexity, because the function of acquiring and releasing the > access key can be served _just as well_ by the mechanism provided by > the constructor and destructor of a scoped object. Assuming a > version of Python that provides the ability to control an object's > lifetime, of course. > > And like I think I mentioned before, it also neatly solves your open > issue regarding existing objects (__exit__ = close) without any code > changes whatsoever. Provided they have properly implemented __del__ > methods, anyway. > >>> In my specific case, poorly written C++ code I can't change that does >>> things in its destructor. >> >> Oh, *ick*. For this case couldn't you implement an extension type >> that >> did roughly >> >> delete self->cpp_object; >> >> in an __exit__ method? > > I did a similar work-around, but I would prefer a version of C++'s > delete with the same semantics in Python if possible. So for this > particular case, I guess I'm really asking for a version of 'del > object' that will always invoke __del__, this is a different issue > than your PEP. I think this is always going to be unlikely... see below. > I think the same exception deal could be applied here too, if > there's another reference when you want the object to go away, throw > an exception. > >>> there's still the issue of programmer error. I'd be nice have an >>> explicit way to say 'this object will no longer exist here'. >> >> I guess I have a lispers approach to this: view memory as infinite >> and assume the implementation does enough behind its back to >> maintain this illusion so long as not too many objects are actually >> live. > > If you seperate the memory and object concepts, an object can cease > to exist even though memory is still allocated for it. Well, this begs another question that I guess underlies all of this: what does it mean for an object to be "alive"? Again, my background is functional/lisp-like languages, and I'm stealing this description more-or-less from Appel's "Modern Compiler Implementation in ML": An object is *live* if it will be used again by the executing program. At any given point of program execution, determining the set of live objects can obviously be at least as hard as the halting problem, so we settle for a weaker notion: An object is *reachable* if it can be reached by traversing the object graph from a given set of roots (in Python, things like locals and the globals of all loaded modules). The runtime pessimistically assumes that all reachable objects are live (and reachable but not live objects can be a cause of memory leaks in Python and other GCed languages). From this point of view, you never actually /destroy/ objects, you just forget about them, and a __del__ method is a side-effect of dropping the last reference to an object -- a slightly unnatural action to have a side effect! This is obviously in contrast to C++, where you /do/ destroy objects (or in the case of local variables, it is clear when the object will be destroyed as a function of program flow). In a sense, the point of the 'with ...' proposal is to get the effect of having scoped access to things like this *without* the conflation of object lifetime and resource access. The notions of "scope" and "extent" as mentioned in Common Lisp the Language: http://www.supelec.fr/docs/cltl/clm/node43.html apply here. > What happens with the memory is an implementation detail, as far as > I'm concerned. I agree obsessing about memory is a distraction. >>> 2) throw an exception at the end of the with block if the object >>> cannot be deleted. >> >> Given that it's the only sane possibility, that's good :-) >> >> Even then, it's probably impossible in Jython, unless I'm missing >> something (and it would seem to permanently limit Python's GC >> choices). > > I noticed this too while I was doing more research last week. Ugh. > > My only thought was similar to the above - seperating the object and > memory concepts. I suspect that Java's JVM ties the two together, > so this may not be possible. I know very little about Java in > general and even less about how the JVM is implemented, though. I don't know a lot about Java, but I hear that experts believe Java's finalizers to be so crippled as to be useless. > How important is compatability w/ Jython? Is it just a happy > coincidence Python can be implemented so well on a JVM, or has it > become a Python design goal? Well, it's become a consideration. If some amazing feature is impossible for Jython but would make Python a much better language, it would probably be accepted. For something that was only a marginal improvement, it might be considered too much a cost. FWIW, I consider my parenthetical remark -- "it would seem to permanently limit Python's GC choices" -- to be more of an issue. > Speaking of research, I also tried to get in touch with Paul but > haven't had any luck. I'll probably post something up on python-dev > shortly. > >> I agree they work well there. >> >> OTOH, I'd feel bad encouraging people to write more __del__ methods >> given the irritating effects they can have (preventing collection >> of cycles being the main one). > > Irritating effects? I'm not familiar with any... preventing > collection of cycles? Not sure what you mean there. If you have a cycle of objects, any one of which has a __del__ method, the cycle collector will refuse to collect it. If your object is referred to (possibly indirectly) by an object in a cycle, it's collection may be deferred almost indefinitely after it ceases to be reachable. > Reading over the Python language specification, I see __del__ is > particularly hairy in general. I'll have to think about it some > more. Java's isn't a model of clarity either: http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Object.html#finalize() The python-dev archives have some pointers to anecdotes about the nasty things you can do to JVMs with finalize() methods (such as limiting the GC to collecting one object per full GC). > Garbage collection puts a whole different spin on these things, and > to date I've spent more time enjoying it than thinking about it. Finalizers in general are a gateway to a world of pain -- you can see this a little bit in C++ with how careful you have to be to not raise exceptions -- but with a GC it only gets worse. No released version of Python is free from problems in this area -- and all I'm confident in saying about the soon-to-be-released 2.3.3 is that I don't know of any. To repeat myself a bit, a motivation for PEP 310 is getting people to write *less* __del__ methods, not more! Cheers, mwh -- 59. In English every word can be verbed. Would that it were so in our programming languages. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html