ARC (Internet Archive)
From Just Solve the File Format Problem
Revision as of 15:03, 30 January 2013 by Dan Tobias (Talk | contribs)
The Internet Archive ARC format is different from and not compatible with the better-known ARC format popular on bulletin board systems in the 1980s. It consists of a series of uncompressed files or data streams (taken directly as received from the Web or another source) combined in a single file with headers giving the original location (URL) from which the data was retrieved, and the archive date.
The Internet Archive ARC format has a successor format, WARC (ISO 28500:2009, Information and documentation -- WARC file format).
By default the Internet Archive crawler, Heritrix, creates records in zipped form, using gzip, with extension arc.gz. An ARC file in zipped form has each ARC record individually zipped and concatenated.
