Release webbase-5.17

*  The home_re table map domain names to regular expressions that
   extract the unique server name. Rules for large free hosting sites
   need this.

*  Signal handling so that the crawl of a URL is never interrupted.
   The exploration will stop after or before loading a URL only.

*  -noheuristics works with -touch to force loading even
   if the content of the URL is in the database.

*  Fix index updating bug that removed documents from the index
   when they are found Not Modified by the crawler.

Release webbase-5.16

*  -version option shows version number

*  Fix allocation error when updating full text index and
   name server timeout condition handling optimization.

*  Use /etc/my.cnf, ~/.my.cnf and datadir/my.cnf instead of
   ~/.my.cnf alone

Release webbase-5.15

*  Implement dynamic updating of the fulltext index.

*  Fix last modified time update bug.

*  Fix mysql-3.23.19a-gamma namespace conflict.

*  Fix bug that left start point in virgin state artificialy.

Release webbase-5.14

*  -crawlers option added to run simultaneous crawlers.

*  signal handling function for graceful interruption of the crawlers.

*  enable url,url_complete and url_content tables to grow over
   4Gb

*  The hook library is now dynamicaly loadable with the -hook option
   so that specific full indexing strategies can be implemented as
   plugins.

*  The -where_url option is taken in account when rebuilding the full
   text index with -rebuild

*  Extensions and mime types have been added to the list of known mime 
   types.

*  The auth field of the start table was removed because it was not used.

Release webbase-5.13

*  The crawler manual page was completely reviewed for correctness.

*  Bug fixes in the mifluz interface.

*  Implemented the -agent option

*  Added the -show option family to display all URLs information
   from an exploration starting point.

*  Improved configuration script.

*  Fixed major leaks and concurency problemes in the langrec
   interface.

*  Widen the scope of allow/disallow comparison to include cgi
   parameters.

*  Restore code to use .my.cnf files if any.

Release webbase-5.12

*  An interface with the langrec library has been added to allow
   language recognition of the crawled documents. The --with-langrec
   configure flag includes the library. By default the library is not
   used. The iso code of the language is stored in the database record
   describing the URL.

*  The documentation has been enriched with many drawings depicting
   the internal architecture of the code.

*  The interface to the full text libaray mifluz has been upgraded to 
   mifluz-0.19

*  Manual pages for consistentc has been added and the man page of
   of crawler reviewed.

*  All known leaks have been fixed.

*  The installation scheme was redesigned to match the GNU/Linux standards

*  The configure script was reviewed and enhanced.

*  Some extensions (.tap, .rar ...) were added to the list of known mime 
   types.

*  A cache for hostnames lookup has been added to reduce the DNS traffic.

*  A usage option was added to the crawler program to replace the terse
   usage.

*  A -where_url option was added to control the scope of URLs given
   to the indexer when rebuilding the full text index (mifluz) from 
   scratch.

*  Filtering of URLs during the crawl can now be made using regular
   expressions.

