I've implemented the frame-recording thing I posted about yesterday. It's now the function annotate.findcreators, and it seems to work *very* well. In fact, I think I prefer it to annotate.groupby: it doesn't have to guess, it seems to give better information and it might work well in a GUI (the information it gives is hierarchical).
And I can't help noticing that the SoC is coming to an end...so I thought I should post about what I've done.
* Everything is split (with the exception of annotate.findcreators at the moment) into Python-implementation-specific (CPython, PyPy, IronPython etc.) and implementation-dependent parts. There's a cpython directory which contains everything CPython-specific.
To get another implementation running, you should only need to write classes to scan instances of object and whatever other types you have that need special treatment, and write functions to get a set of roots and to get the name of a type.
* The scanner works. It's written in an implementation-independent way (I hope). If the implementation supports type.mro(), it can construct a wrapper of a type from the wrappers of its superclasses - i.e. given code to scan class Foo and class Bar, it can scan class Baz(Foo, Bar).
In practice this means that only scanning code for very strange objects (in the CPython implementation, C objects) needs to be defined, and the scanner will sort out the rest itself.
The scanner gets wrapper classes from the module sizes, which should be filled in by the implementation-specific part, and which can also be used to treat some objects and types specially. For example, the scanner puts an entry in here which causes no wrappers to be counted as taking space.
* The code is quite useful, with a fair amount of ways of analysing the scanner's output (including making graphs :-). In a short time I found some waste of memory in SCons and in about half an hour reduced its memory usage from 24MB to 22MB. (I haven't tried to reduce it any further yet - there's certainly room for improvement.)
* There are (hopefully useful) docstrings on all public functions and classes. I've written part of a tutorial, which has links to HappyDoc-generated docs made from the docstrings, and a document describing the internals of the scanner.
* The CPython-specific code can scan all built-in types which are at all possible to scan.
On the other hand:
* I haven't written any code to scan any types from extension modules. I will do when it becomes a problem - lots of programs don't use extensions much.
* There's a bit of a mess when making copies of wrappers. I think I can fix that by adding an extra level of indirection.
* Each wrapper contains a reference to the object it's wrapping, and quite a lot of code uses it. That's not very good for mutable types, since the object can have changed when code looks at it, and it also prevents pickling the sets of wrappers for future use.
* There's no nice GUI yet...once I've fixed the problems with wrappers, I think I'll have a go at writing one.
Also, I want to port it to other implementations. PyPy looks very tempting :-)
![[Comments]](/crew/mwh/blog//themes/Default/comments.png "1 comment")
(1) Thu Sep 01 2005 22:34 CET-1CEST:
under a Creative Commons License.