[triangle-zpug] Python spider
tbryan at python.net
Sat Jul 10 19:47:15 CEST 2004
On Wednesday 07 July 2004 04:31 pm, David Chandek-Stark wrote:
> Has anyone written a spider in Python -- or know of one -- that would be
> useful in "dumping" a Zope site to the file system? I'm working with
> Wget now, and it may be sufficient for my needs, but I'd like to know
> that I could fall back on a customized Python solution, if necessary.
The Python distribution comes with (or, it used to come with) a utility called
webchecker. If you're on a UNIX system, and you built Python from source,
you'll find it under the Tools directory. For example,
The websucker.py script can be used to spider a site/create a local copy of a
site. It has failed on some sites, but it has generally suited my needs when
I needed something like this.
More information about the triangle-zpug