There are a couple different ways to mirror the Apache Software
Foundation software distributions. Here is the list of ways we make
available. We strongly recommend that you use rsync. Other techniques may
or may not work.
rsync
You can find details on rsync at http://rsync.samba.org/. You
might want to call it using the following arguments, for example:
rsync -rtlzv --delete --exclude incoming
www.apache.org::apache-dist /local/path/to/mirror
You can also just do a rsync www.apache.org:: to get a
list of rsync modules available.
CVSup
This is a package which uses the versioning scheme in CVS to determine
exactly which parts of which files changed between accesses, and is thus
by far the most efficient mechanism for transferring CVS-related files.
The best place to find out more about CVSup is at the
CVSup FAQ. There are binaries for Linux, *BSD, Solaris, SunOS,
Alpha/OSF-1, HPPA-10.20, and Irix 6.
After you've set up the client binary, use the following as a "supfile":
*default host=www.apache.org
*default base=(wherever you want)
*default prefix=(wherever you want)
*default release=cvs
apache-fullsite
Other Techniques
We do not recommend the use of Web spiders, or HTTP equivalents to the
"mirror.pl" package, as every object needs to get a HEAD request every
time it is checked, which is a very inefficient mechanism for seeing
what's been updated on an entire site.
It is not possible to mirror via ftp. The other methods are also far
more efficient so you will be pleased after setting them up.