makehistory



MAKEHISTORY(8)            InterNetNews Documentation            MAKEHISTORY(8)




NAME

       makehistory - Initialize or rebuild INN history database


SYNOPSIS

       makehistory [-abeFIOx] [-f filename] [-l count] [-T tmpdir] [-s size]


DESCRIPTION

       makehistory rebuilds the history(5) text file, which contains a list of
       Message-IDs of articles already seen by the server.  It can also be
       used to rebuild the overview database.  Note that the dbz(3) indexes
       for the history file are rebuilt by makedbz(8), not by makehistory as
       in earlier versions of INN.

       The default location of the history text file is pathdb/history; to
       specify an alternate location, use the -f flag.

       By default, makehistory will scan the entire spool, using the storage
       manager, and write a history line for every article.  To also generate
       overview information, use the -O flag.

       WARNING: If you’re trying to rebuild the overview database, be sure to
       stop innd(8) and delete or zero out the existing database before you
       start for the best results.  An overview rebuild should not be done
       while the server is running.  Unless the existing overview is deleted,
       you may end up with problems like out-of-order overview entries, exces-
       sively large overview buffers, and the like.

       If ovmethod in inn.conf is "ovdb", you must have the ovdb processes
       running while rebuilding overview.  ovdb needs them available while
       writing overview entries.  You can start them by hand separate from the
       rest of the server by running ovdb_init; see ovdb_init(8) for more
       details.


OPTIONS

       -a  Append to the history file rather than generating a new one.  If
           you append to the main history file, make sure innd(8) is throttled
           or not running, or you can corrupt the history.

       -b  Delete any messages found in the spool that do not have valid Mes-
           sage-ID headers in them.

       -e  Compute Bytes headers which is used for overview data.  This option
           is valid only if -O flag is specified and overview.fmt includes
           "Bytes:".

       -f filename
           Rather than writing directly to pathdb/history, instead write to
           filename.

       -F  Fork a separate process to flush overview data to disk rather than
           doing it directly.  The advantage of this is that it allows make-
           history to continue to collect more data from the spool while the
           first batch of data is being written to the overview database.  The
           disadvantage is that up to twice as much temporary disk space will
           be used for the generated overview data.  This option only makes
           sense in combination with -O.  With "buffindexed", the "overchan"
           program is invoked to write overview.

       -I  Don’t store overview data for articles numbered lower than the low-
           est article number in active.  This is useful if there are for
           whatever reason old articles on disk that shouldn’t be available to
           readers or put into the overview database.

       -l count
           This option specifies how many articles to process before writing
           the accumulated overview information out to the overview database.
           The default is 10000.  Since overview write performance is faster
           with sorted data, each "batch" gets sorted.  Increasing the batch
           size with this option may further improve write performance, at the
           cost of longer sort times.  Also, temporary space will be needed to
           store the overview batches.  At a rough estimate, about 300 * count
           bytes of temporary space will be required (not counting temp files
           created by sort(1)).  See the description of the -T option for how
           to specify the temporary storage location.  This option has no
           effect with "buffindexed", because "buffindexed" does not need
           sorted overview and no batching is done.

       -s size
           Size the history database for approximately size pairs.  Accurately
           specifying the size is an optimization that will create a more
           efficient database.  (The size should be the estimated eventual
           size of the history file, typically the size of the old file, in
           lines.)

       -O  Create the overview database as well as the history file.  Overview
           information is only required if the server supports readers; it is
           not needed for a transit-only server (see enableoverview in
           inn.conf(5)).  If you are using the "buffindexed" overview storage
           method, erase all of your overview buffers before running makehis-
           tory with -O.

       -T tmpdir
           If -O is given, makehistory needs a location to write temporary
           overview data.  By default, it uses pathtmp, set in inn.conf, but
           if this option is given, the provided tmpdir is used instead.  This
           is also used for temporary files created by sort(1) (which is
           invoked in the process of writing overview information since sorted
           overview information writes faster).  By default, sort usually uses
           your system temporary directory; see the sort(1) man page on your
           system to be sure.

       -x  If this option is given, makehistory won’t write out history file
           entries.  This is useful mostly for building overview without gen-
           erating a new history file.


EXAMPLES

       Here’s a typical example of rebuilding the entire history and overview
       database, removing broken articles in the news spool.  This uses the
       default temporary file locations and should be done while innd isn’t
       running (or is throttled).

           makehistory -b -f history.n -O -l 30000 -I

       This will rebuild the overview (if using "buffindexed", erase the
       existing overview buffers before running this command) and leave a new
       history file as "history.n" in pathdb.  To preserve all of the history
       entries from the old history file that correspond to rejected articles
       or expired articles, follow the above command with:

           cd /usr/local/news/db
           awk ’NF == 2 { print }’ < history >> history.n

       (replacing the path with your pathdb, if it isn’t the default).  Then
       look over the new history file for problems and run:

           makedbz -s ‘wc -l < history‘ -f history.n

       Then rename all of the files matching "history.n.*" to "history.*",
       replacing the current history database and indexes.  After that, it’s
       safe to unthrottle innd.

       For a simpler example:

           makehistory -b -f history.n -I -O

       will scan the spool, removing broken articles and generating history
       and overview entries for articles missing from history.

       To just rebuild overview:

           makehistory -O -x -F


FILES

       inn.conf
           Read for pathdb, pathtmp, and other settings.

       pathdb/history
           This is the default output file for makehistory.

       pathtmp
           Where temporary files are written unless -T is given.


HISTORY

       Originally written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews
       and updated by various other people since.

       $Id: makehistory.8,v 1.37.2.1 2003/07/12 19:26:57 rra Exp $


SEE ALSO

       dbz(3), active(5), history(5), inn.conf(5), ctlinnd(8), innd(8),
       makedbz(8), ovdb_init(8), overview.fmt(5).



INN 2.4.1                         2003-07-05                    MAKEHISTORY(8)

Man(1) output converted with man2html