Compression

From Just Solve the File Format Problem
(Difference between revisions)
Jump to: navigation, search
(→‎Compression schemes used in multiple formats)
(→‎Compression algorithms and compressed data formats)
 
(129 intermediate revisions by 7 users not shown)
Line 2: Line 2:
 
|formattype=electronic
 
|formattype=electronic
 
|thiscat=Compression
 
|thiscat=Compression
 +
|image=Vise.png
 
}}
 
}}
  
Line 8: Line 9:
 
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive.
 
A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like [[tar]] can be combined with an archive format to produce a proper compressed archive.
  
−
== Compression schemes used in multiple formats ==
+
== Compression algorithms and compressed data formats ==
 +
 
 +
(excluding formats usually used as file formats)
  
 
* [[Arithmetic coding]]
 
* [[Arithmetic coding]]
 +
* [[Brotli]]
 +
* [[Burrows–Wheeler transform]]
 
* [[DEFLATE]]
 
* [[DEFLATE]]
 +
* [[DivANS]]
 +
* [[EXEPACK2]]
 +
* [[FastLZ]]
 +
* [[Fractal compression]]
 +
* [[HPACK (HTTP)|HPACK]] (used to compress HTTP headers)
 
* [[Huffman coding]]
 
* [[Huffman coding]]
−
* [[Lempel-Ziv]]
+
** [[Adaptive Huffman coding]]
 +
** [[Canonical Huffman code]]
 +
* [[InstaCompOne]]
 +
* [[Lempel–Ziv–Stac]] (LZS)
 +
* [[LZ4]]
 +
* [[LZ77]]
 +
* [[LZ77 with Huffman coding]]
 +
* [[LZ78]]
 +
* [[LZFSE]]
 +
* [[LZHAM]]
 +
* [[LZMA]]
 +
* [[LZMA2]]
 +
* [[LZMW]]
 +
* [[LZO]] (Lempel–Ziv–Oberhumer)
 +
* [[LZRW]]
 +
* [[LZSS]] (Lempel–Ziv–Storer–Szymanski)
 
* [[LZW]]
 
* [[LZW]]
 
* [[LZX]]
 
* [[LZX]]
 +
** [[LZX Delta]]
 +
* [[MSZIP]]
 +
* [[PKWARE DCL Implode]]
 +
* [[PPMd]]
 +
* [[Prediction by partial matching]] (PPM)
 
* [[Run-length encoding]]
 
* [[Run-length encoding]]
 +
** [[PackBits]]
 +
** [[RLE90]]
 +
* [[Shannon–Fano coding]]
 +
* [[Snappy]] (Google-created compression also used by Apple in iWork formats)
 +
* [[zlib]]
 +
* [[Zstandard]] (Zstd)
 +
** [[Zstandard dictionary]]
  
−
== Specific formats/programs ==
+
=== Graphics compression ===
 +
See [[Graphics#Compression]] for compression formats used primarily with graphics.
 +
 
 +
== Specific file formats/programs ==
 +
 
 +
(multi-file compressors are in [[Archiving]])
  
−
* [[7z]]
 
 
* [[9CDR]] (Amiga FileImploder Clone)
 
* [[9CDR]] (Amiga FileImploder Clone)
 +
* [[ABC (Advanced Blocksorting Compressor)]]
 +
* [[ABComp]] (.abp)
 +
* [[Aldus PKZP]]
 
* [[BARF]] (.x, .x??)
 
* [[BARF]] (.x, .x??)
 +
* [[BGZF]]
 +
* [[BWTC32Key]] (.B3K)
 
* [[bzip]] (.bz)
 
* [[bzip]] (.bz)
 
* [[bzip2]] (.bz2)
 
* [[bzip2]] (.bz2)
−
* [[compress]] (.Z)
+
* [[BZip3]] (.bz3)
−
* [[CCITT Group 4]]
+
* [[CAZIP]]
 +
* [[ChiefLZ]] (.LZZ)
 +
* [[compact (Gene Olson)]] (.W)
 +
* [[compact (Unix)]] (.C)
 +
* [[compress (Unix)]] (.Z)
 +
* [[CMP (NEW-DOS)]]
 
* [[CrLZH]] (.?y?)
 
* [[CrLZH]] (.?y?)
 
* [[CRN]] (.crn) - compressed text files used for PC-Write manual
 
* [[CRN]] (.crn) - compressed text files used for PC-Write manual
 
* [[Crunch]] (.?z?)
 
* [[Crunch]] (.?z?)
−
* [[DiskDoubler]]  
+
* [[Crunch-Mania]]
 +
* [[DACT]]
 +
* [[DitPack]]
 
* [[DUPA]] (Amiga FileImploder Clone)
 
* [[DUPA]] (Amiga FileImploder Clone)
 +
* [[EDI Install archive]]
 +
* [[EDI Install packed file]]
 +
* [[EDI LZSSLib]]
 +
* [[EmmaCompress]]
 
* [[Error Code Modeler]] (.ecm)
 
* [[Error Code Modeler]] (.ecm)
 
* [[File Imploder]] (Amiga) (.imp)
 
* [[File Imploder]] (Amiga) (.imp)
 +
* [[FlySpeed st/exp]]
 
* [[Freeze/Melt]] (Unix) (.F)
 
* [[Freeze/Melt]] (Unix) (.F)
 
* [[gzip]] (.gz)
 
* [[gzip]] (.gz)
 +
* [[Hammer (compression)|Hammer]]
 +
* [[HAP]] (v2.x)
 +
* [[ICE (Michael Lamanuzzi)|ICE]] (.ice, by Michael Lamanuzzi)
 +
* [[Inflate]] (.infl) - parody format that actually increases size
 +
* [[InstallIt!]]
 +
* [[JAM (W. Jiang)]]
 +
* [[The JAM Packer|JAM Packer, The]]
 
* [[JCalG1]] (.jc, Commodore Amiga)
 
* [[JCalG1]] (.jc, Commodore Amiga)
−
* [[LZIP]] (.lz)
+
* [[Kboom11]]
−
* [[LZMA]] (.lzma)
+
* [[lrzip]] (.lrz)
−
* [[LZOP]] (.lzop)
+
* [[LZ5 and Lizard]]
−
* [[pack]] (.z)
+
* [[LZARI]]
−
* [[RK]] (WinRK)
+
* [[LZHUF]]
 +
* [[Lzip]] (.lz)
 +
* [[LZMA Alone|LZMA_Alone]] (.lzma)
 +
* [[lzop]] (.lzo)
 +
* [[LZS221]]
 +
* [[LZSS (Haruhiko Okumura)]]
 +
* [[LZWCOM]]
 +
* [[LzwPak]]
 +
* [[Microsoft KWAJ installation compression]]
 +
* [[Microsoft SZ installation compression]]
 +
* [[Mozilla LZ4]] (.jsonlz4)
 +
* [[MS-DOS installation compression]] (.??_)
 +
* [[MULTIKIT COMPRESS]]
 +
* [[NetWare Packed File]]
 +
* [[NOAH]]
 +
* [[Nova Compress]]
 +
* [[NPack]]
 +
* [[NRV (Not Really Vanished)]]
 +
* [[Oodle Data Compression]]
 +
* [[pack (Unix)]] (.z)
 +
* [[PGNPack]]
 +
* [[PowerPacker]] (.pp)
 +
* [[PPMZ]]
 +
* [[PPMZ2]]
 +
* [[Pro-Pack]]
 +
* [[PUT and GET (Scott Jennings)]]
 +
* [[Quasijarus Strong Compression]]
 +
* [[RAC]]
 +
* [[Reduq]]
 +
* [[RKUC]]
 +
* [[rzip]] (.rz)
 +
* [[S2]] (.s2, .snappy)
 +
* [[SCO compress LZH]]
 +
* [[SEMONE]] (.one)
 +
* [[sfArk]]
 +
* [[SFDN Packer]]
 +
* [[SHARC]] (density)
 
* [[Softdisk Text Compressor]] (.ctx)
 
* [[Softdisk Text Compressor]] (.ctx)
 +
* [[Splint]]
 +
* [[SQ2]]
 
* [[Squash (RISC OS)|Squash]] - single file compression on RISC OS
 
* [[Squash (RISC OS)|Squash]] - single file compression on RISC OS
 
* [[Squeeze|Squeeze/SQ]] (.?q?)
 
* [[Squeeze|Squeeze/SQ]] (.?q?)
 +
* [[SQUEEZE (AJP Systems)]]
 +
* [[squeeze (Daniel Bernstein)]] (.MW)
 +
* [[Squish (Mike Albert)]]
 +
* [[Sunzip]] (.sz)
 +
* [[TERSE]]
 +
* [[Top4]]
 +
* [[TTCOMP]]
 +
* [[TTW]]
 +
* [[tzip]]
 +
* [[Tzip (Text ZIP)]]
 +
* [[UHBC]]
 +
* [[WWPACK]]
 +
* [[XPK]]
 
* [[XZ]] (.xz)
 
* [[XZ]] (.xz)
−
* [[zlib]]
+
* [[Yaz0]]
 +
* [[zchunk]]
 +
* [[Zoo filter format]]
 +
* [[Zoo Z format]]
 +
* [[ZSQ (LZW compression)]]
 +
 
 +
== Transparent file compression ==
 +
(A mostly-obsolete technique in which a background process hooks into the OS and decompresses files transparently when they are read. ''Compression'' might be done manually. The line between this and ''Disk compression'' can be blurry.)
 +
 
 +
* [[AFX (Atari ST)]]
 +
* [[CFS (Computer Concepts Compression)]]
 +
* [[DIET (compression)|DIET]]
 +
* [[Diet Disk]]
 +
* [[Pack-Ice]]
 +
* [[SLIM (Dominic Herity)|SLIM]]
 +
* [[Turbo Packer]]
 +
* [[XPACK]]
 +
 
 +
== Disk compression ==
 +
(Transparent disk compression software and the like, excluding filesystem formats.)
 +
 
 +
* [[DiskDoubler]]
 +
* [[DriveSpace]] and DoubleSpace
 +
* [[JAM (disk compression)]]
 +
* [[Stacker]]
 +
 
 +
For filesystem formats that feature compression, see [[Filesystem]] (especially [[Filesystem#Compressed filesystems]]).
 +
 
 +
== Executable compression ==
 +
 
 +
See [[Executable compression]].
  
 
== See also ==
 
== See also ==
  
 +
* [[Compression and archiving software]]
 +
* [[Compression research]]
 
* [[Archiving]] (for formats that place multiple files together in one file, with or without compression)
 
* [[Archiving]] (for formats that place multiple files together in one file, with or without compression)
 +
* [[Graphics#Compression|Graphics]] (for image-specific compression)
 +
* [[Video]] (for video-specific compression)
 +
* [[Audio and Music]] (for audio-specific compression)
 +
 +
== Links ==
 +
* [http://boingboing.net/2014/03/04/data-compression-with-playing.html Data-compression with playing cards]
 +
* [http://www.quora.com/What-is-the-most-compressed-file-ever/answer/Kelly-Martin?srid=uejz&share=1 What is the most compressed file ever?]
 +
* [http://ieeeghn.org/wiki/index.php/History_of_Lossless_Data_Compression_Algorithms History of Lossless Data Compression Algorithms]

Latest revision as of 12:18, 27 April 2024

File Format
Name Compression
Ontology

{{{caption}}}

Contents

[edit] Stream compression formats

A stream format takes a stream of bytes, and outputs a different, hopefully smaller, stream of bytes. These compression formats are often used internally in other data structures to compress data, as well as in network protocols, such as http. Used stand-alone, a stream compression format does not offer archiving capability, however in the UNIX doctrine, an archiver like tar can be combined with an archive format to produce a proper compressed archive.

[edit] Compression algorithms and compressed data formats

(excluding formats usually used as file formats)

[edit] Graphics compression

See Graphics#Compression for compression formats used primarily with graphics.

[edit] Specific file formats/programs

(multi-file compressors are in Archiving)

[edit] Transparent file compression

(A mostly-obsolete technique in which a background process hooks into the OS and decompresses files transparently when they are read. Compression might be done manually. The line between this and Disk compression can be blurry.)

[edit] Disk compression

(Transparent disk compression software and the like, excluding filesystem formats.)

For filesystem formats that feature compression, see Filesystem (especially Filesystem#Compressed filesystems).

[edit] Executable compression

See Executable compression.

[edit] See also

[edit] Links

Retrieved from "http://fileformats.archiveteam.org/index.php?title=Compression&oldid=48501"
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox