Change Log

Version 1.2--October, 2002

1) Corrected bugs.
------------------

1.a) In module bpcca.py, if the second field was not two-dimensional, the
     program crashed. It is already corrected.

1.b) The last character in the last line of a file could be missed by
     readdat.py if the last line did not end in a new line. It is corrected now.

2) Updated modules and additions
--------------------------------

2.1) Added a netCDF iterator. This is an object which reads

a list of filenames (all of the COARDS-compliant netCDF type) and is able to
iterate through all the records in the list of files. It has methods like
__len__ and __getitem__ and  getfield(), to handle the iterator and the fields
inside. It is a cheap version of ncrcat. Its main interest is that under some
systems (Linux, for instance), files larger than 2 Gb may produce errors in
the filesystem. This way, they may be processed as smaller chunks and, still,
give the user the impression of a single object that can be traversed by means
of a single iteration.

2.2) Huge dataset EOFs (hdseofs.py)

Is a version of the EOF computing-code for huge datasets. Instead of
performing the SVD of the data matrix, like svdeofs.py does, it simply
iterates once or twice over the dataset, depending on the type instantiated.
It builds the covariance matrix and performs a SVD of it. Then, after
iterating once more over the dataset, it can compute the PCs. Other bells and
whistles, like Monte Carlo tests and the North test or an automatic
shape/unshape mechanism of the eigenvectors or even the possibility of the
call of a external SVD solver (not the one in LapackLite) are also included.
It is useful for Gigabyte-sized data, as these problems are difficult to solve
by the previously available EOF-solver (svdeofs.py) due to memory constraints.
In order to efficiently manage the memory, it uses the "memory-managing"
versions of Numeric.add(), Numeric.reduce() and so on. However, it is slower
than the previous version of the routines and should only be used when the
dataset is very big.

2.3) Monte Carlo test for the Barnett-Preisendorfer version of the CCA

We have added a new Monte carlo test on the congruence coefficient of
canonical correlation patterns of random subsamples of the canonical
correlation analysis of fields by means of the Barnett-Preisendorfer approach.

2.4) Search of analogs for downscaling purposes

We perform the search in the PCA space as well as in the CCA space, too.
Mathematical details can be found in (Fernandez and Saenz, under revision).

2.5) Array of numerical histograms (NHArray)

A special module has been written in C to allow the use of arrays of numerical
histograms for Monte Carlo tests.  It is much faster then the iteration over a
multivariate distribution of univariate histograms as those available in
Scientific.Statistics.

2.6) svdeofs.py

A new object-oriented front end to the EOFs computation. Some methods have
been added, for instance, take a look in the manual at recontructedField,
unreconstructedField, projectField

2.7) Cleaner name spaces

We have removed ALMOST all the imports of the kind from XXXX import * This
means, basically, that our code is cleaner and that it will not (hopefully)
corrupt your name spaces, but it means more requirements on your code as well.
It can also create some incompatibilities with older versions of PyClimate.
However, in order to leave the user name space cleaner, we have adhered to
these stricter requirements.

The next script:
#####################
from pyclimate.ncstruct import *

onc=create_bare_COARDS("XXX.nc",None,None,arange(0.,377.6,2.5),None)
#####################

would work in PyClimate 1.1, but it will crash in PyClimate 1.2.  It worked
before because Numeric was in your name space because pyclimate.ncstruct
imported it and functions such as 'arange' were automatically in your
namespace. Under PyClimate 1.2, this is no longer true.

To make the script run under PyClimate 1.2, you must add line of code to
import Numeric yourself

#####################
from pyclimate.ncstruct import *
from Numeric import * # Add this line

onc=create_bare_COARDS("XXX.nc",None,None,Numeric.arange(0.,377.6,2.5),None)
#####################

In particular, versions 1.1 and 1.2 of PyClimate allow the user to call
equally pyclimate.readdat, pyclimate.writedat and pyclimate.asciidat. This is
an obsolescent feature. In a couple of versions (about twenty years in the
future), pyclimate.asciidat will be the only way to access all these functions
and pyclimate.readdat and pyclimate.writedat will disappear.

2.8) Statistical tests added to tools.py

Two new functions are available in the module pyclimate.tools. 'ttest'
performs a t-Student test on the difference of means and 'ftest' performs a
F-test on the difference of variances.

2.9) writedat.py formatstring argument has changed a bit

Blank spaces are managed by the writedat function, there's no need 
to worry about them in the 'formatstring' argument.

Jon Saenz and Jesus Fernandez, December 2002.