[MMTK] Fwd: scientific python

[hinsen at llb.saclay.cea.fr]

On Oct 4, 2004, at 10:16, Vincent Kraeutler wrote:

> given that
> -- many people seem to have trouble with python/netcdf,

That seems to be a problem exclusive to Windows users. One possibility 
would be not to rely on a netCDF installation under Windows and include 
the required parts of the netCDF source code for Windows into the 
ScientificPython distribution. This is not as much work as it sounds, 
because there is a minimal one-file version of netCDF 3.4 floating 
around on the Internet, I have used it for the Sharp Zaurus, for 
example. But the integration would have to be done by someone who is 
competent about Windows, i.e. not me. Another consideration is 
licensing of course.

Note that this cannot be a general approach for all platforms, as 
netCDF has its own platform dependencies that are checked during 
installation.

> -- and biosimgrid uses python binary pickles as native trajectory 
> format
>    (comparable in both size and access speed it seeems),
> -- mmtk seems to use netcdf only for trajectory storage,
> would it seem reasonable to propose the use of binary pickles for
> mmtk trajectories as well? while i've been lurking this mailing list

I don't think that python pickles make a good trajectory format, 
because they have to be read into memory in one block. That excludes 
the use of trajectories that are larger than available memory. One 
could envisage the use of the shelve module, but there are still some 
disadvantages: only one access mode (by time step or by atom), data 
accesible only through Python.

The reasons why I picked netCDF are the following:

1) Portability of binary data
2) Self-explanatory data (everything in one file)
3) Good support of the format by many languages
4) Reasonably efficient access to arbitrary subsets of the data

I am not aware of any other trajectory format in common use that 
satisfier criterion 4), which is why I ended up inventing my own.

The only alternative to netCDF that offers the same advantages is HDF. 
Back when I made the choice, netCDF was ahead, but today I might prefer 
HDF5 (not enough though to consider switching).

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Laboratoire Léon Brillouin, CEA Saclay,
91191 Gif-sur-Yvette Cedex, France
Tel.: +33-1 69 08 79 25
Fax: +33-1 69 08 82 61
E-Mail: hinsen at llb.saclay.cea.fr
---------------------------------------------------------------------