EBZip
(Updated magic.) |
(Updated magic, after comparing it with other .ebz files like HONMON.ebz and GAI*.ebz files.) |
||
(2 intermediate revisions by one user not shown) | |||
Line 3: | Line 3: | ||
|subcat=Compression | |subcat=Compression | ||
|extensions={{ext|ebz}} | |extensions={{ext|ebz}} | ||
− | |||
|compression=lossless | |compression=lossless | ||
− | |magic=45 42 | + | |magic=45 42 5A 69 70 15 00 00 00 00 |
|released=1999<ref>[https://web.archive.org/web/20010619200710/http://www.sra.co.jp/people/m-kasahr/eb/ChangeLog-ja.html ChangeLog - SRA website mirrored on Internet Archive]</ref> | |released=1999<ref>[https://web.archive.org/web/20010619200710/http://www.sra.co.jp/people/m-kasahr/eb/ChangeLog-ja.html ChangeLog - SRA website mirrored on Internet Archive]</ref> | ||
}} | }} | ||
{{notice|1=Majority of the links/citations are in Japanese.}} | {{notice|1=Majority of the links/citations are in Japanese.}} | ||
− | '''Electronic Book Zip or EBZip''' is a compression format used in electronic dictionaries that were distributed primarily on compact discs (CD) | + | '''Electronic Book Zip or EBZip''' is a compression format used in electronic dictionaries that were distributed primarily on compact discs (CD). It is also one of the many frontend utilities that uses the EB library for manipulating [[EPWING]] format. |
− | The | + | The electronic dictionaries stored on computers eliminates the need to have physical media, and allows remote referencing if it was connected to a network, using the likes of <code>dserver</code> or <code>ebnetd</code> for instance. It is likely that compression was needed due to the fact that hard disks were expensive compared to compact discs, when having a number of electronic dictionaries stored on a computer, it made little sense to retain physical media just to reference a number of words every so often, and to juggle around ejecting/inserting discs into the computer. For the expense of sacrificing few CPU cycles and potentially insignificant amount of memory, these electronic dictionaries could be made to take less storage space<ref>[https://web.archive.org/web/20170705202139/http://sci.lang.japan.narkive.com/X84h0clT/kenkyusha-dictionary-cd-and-epwing#post2 Kenkyusha dictionary CD and EPWING - sci.lang.japan NARKIVE - Internet Archive copy]</ref>, and allows faster dictionary lookups due to hard disks allowing random access. |
− | + | The compression is lossless, initially using a modified [https://www.zlib.net/ zlib]<ref>[https://github.com/mistydemeo/eb/blob/742248c23cf7769d0fed412cbce458222a437795/zlib/README.altered Last version released with internal zlib - README.altered - version 4.1.3 - Misty De Méo]</ref>, then likely unmodified zlib by the original <code>ebzip</code> author<ref>[https://github.com/mistydemeo/eb/commit/742ddab4f255b7c34d7d1f1b4e3fd245bfc3ea18 Internal zlib removed - Commit 742ddab (4.2) - Misty De Méo]</ref>. Do keep in mind that not everything is compressed, EBZ format is not a container like [[ZIP]] for instance, selected files (typically <code>CATALOGS</code> and <code>HONMON</code>) are compressed and have their filenames bearing these extensions, e.g. <code>HONMON.ebz</code>. | |
== History == | == History == | ||
− | |||
− | |||
EB-like software for Unix-like began circulating on Japanese Usenet as early as September 1991 in the form of [[shar]] by Shingo Nishioka of Osaka University.<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=NISHIOKA.91Sep3191513@sp4.sanken.osaka-u.ac.jp dserver/Electronic Book Dictionary server & clients (1/2) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=NISHIOKA.91Sep3191550@sp4.sanken.osaka-u.ac.jp dserver/Electronic Book Dictionary server & clients (2/2) - AncientFJ - katsu.watanabe.name]</ref> Versions before 2.1beta of <code>dserver</code> lacked compression, which was introduced as early as 1996-03-25 by Yura Shunsuke.<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=4j5h0e$2am@isnews.is.s.u-tokyo.ac.jp dserver 2.1-beta patch for compressed dictionary - AncientFJ - katsu.watanabe.name]</ref> Gradually over time, Motoyuki Kasahara, a member of Software Research Associates (SRA) and the author of EB Library, began developing <code>ndtpd</code> which handles electronic dictionary or dictionaries over the network, replacing <code>dserver</code>.<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5rp4qp$3sk$1@sranhh.sra.co.jp BookView 1.0 - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sctc3$irv$2@sranhh.sra.co.jp BookView 1.0.2 (Re: BookView 1.0) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=64emm8$mcf$1@sranhh.sra.co.jp BookView 1.1 (1/3) - AncientFJ - katsu.watanabe.name]</ref> The first release of such software was around 1997-06-04 as <code>ndtpd-1.0beta0</code> (via FTP) with which it included <code>eb</code> which was a library. Earliest release on fj.sources was 1997-08-07 as <code>ndtpd-1.0</code>, which also describes the relationship between <code>dserver</code> and <code>ndtpd</code>, as [[shar]].<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4ns$31a$2@sranhh.sra.co.jp ndtpd 1.0 (0/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4pr$3o1$1@sranhh.sra.co.jp ndtpd 1.0 (1/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4rn$3o1$2@sranhh.sra.co.jp ndtpd 1.0 (2/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4te$3o1$3@sranhh.sra.co.jp ndtpd 1.0 (3/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4vf$3o1$4@sranhh.sra.co.jp ndtpd 1.0 (4/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc514$3o1$5@sranhh.sra.co.jp ndtpd 1.0 (5/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc52p$3o1$6@sranhh.sra.co.jp ndtpd 1.0 (6/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc54s$3o1$7@sranhh.sra.co.jp ndtpd 1.0 (7/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc55j$3o1$8@sranhh.sra.co.jp ndtpd 1.0 (8/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc56d$3o1$9@sranhh.sra.co.jp ndtpd 1.0 (9/9) - AncientFJ - katsu.watanabe.name]</ref> Until the release of <code>ndtpd-3.0alpha0</code> (2000-06-04), <code>eb</code> has always been included. EB Library began as a separate project with the first release in 1999-06-16 as <code>eb-2.2</code>. Both of these projects continued until the discontinuation of <code>ndtpd</code> (as a release) on 2003-05-24 with the version 3.1.5. <code>eb</code> remained to be maintained until 2010-03-08 with the version 4.4.3. | EB-like software for Unix-like began circulating on Japanese Usenet as early as September 1991 in the form of [[shar]] by Shingo Nishioka of Osaka University.<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=NISHIOKA.91Sep3191513@sp4.sanken.osaka-u.ac.jp dserver/Electronic Book Dictionary server & clients (1/2) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=NISHIOKA.91Sep3191550@sp4.sanken.osaka-u.ac.jp dserver/Electronic Book Dictionary server & clients (2/2) - AncientFJ - katsu.watanabe.name]</ref> Versions before 2.1beta of <code>dserver</code> lacked compression, which was introduced as early as 1996-03-25 by Yura Shunsuke.<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=4j5h0e$2am@isnews.is.s.u-tokyo.ac.jp dserver 2.1-beta patch for compressed dictionary - AncientFJ - katsu.watanabe.name]</ref> Gradually over time, Motoyuki Kasahara, a member of Software Research Associates (SRA) and the author of EB Library, began developing <code>ndtpd</code> which handles electronic dictionary or dictionaries over the network, replacing <code>dserver</code>.<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5rp4qp$3sk$1@sranhh.sra.co.jp BookView 1.0 - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sctc3$irv$2@sranhh.sra.co.jp BookView 1.0.2 (Re: BookView 1.0) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=64emm8$mcf$1@sranhh.sra.co.jp BookView 1.1 (1/3) - AncientFJ - katsu.watanabe.name]</ref> The first release of such software was around 1997-06-04 as <code>ndtpd-1.0beta0</code> (via FTP) with which it included <code>eb</code> which was a library. Earliest release on fj.sources was 1997-08-07 as <code>ndtpd-1.0</code>, which also describes the relationship between <code>dserver</code> and <code>ndtpd</code>, as [[shar]].<ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4ns$31a$2@sranhh.sra.co.jp ndtpd 1.0 (0/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4pr$3o1$1@sranhh.sra.co.jp ndtpd 1.0 (1/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4rn$3o1$2@sranhh.sra.co.jp ndtpd 1.0 (2/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4te$3o1$3@sranhh.sra.co.jp ndtpd 1.0 (3/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc4vf$3o1$4@sranhh.sra.co.jp ndtpd 1.0 (4/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc514$3o1$5@sranhh.sra.co.jp ndtpd 1.0 (5/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc52p$3o1$6@sranhh.sra.co.jp ndtpd 1.0 (6/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc54s$3o1$7@sranhh.sra.co.jp ndtpd 1.0 (7/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc55j$3o1$8@sranhh.sra.co.jp ndtpd 1.0 (8/9) - AncientFJ - katsu.watanabe.name]</ref><ref>[https://katsu.watanabe.name/ancientfj/article.php?mid=5sc56d$3o1$9@sranhh.sra.co.jp ndtpd 1.0 (9/9) - AncientFJ - katsu.watanabe.name]</ref> Until the release of <code>ndtpd-3.0alpha0</code> (2000-06-04), <code>eb</code> has always been included. EB Library began as a separate project with the first release in 1999-06-16 as <code>eb-2.2</code>. Both of these projects continued until the discontinuation of <code>ndtpd</code> (as a release) on 2003-05-24 with the version 3.1.5. <code>eb</code> remained to be maintained until 2010-03-08 with the version 4.4.3. | ||
− | + | == EBUZip == | |
+ | EBUZip is a fork of EBZip that is maintained by Ito Kazuhiro. The 'U' in EBUZip denotes the ability to support JIS X 4081 UTF-8 extensions provided that it is compiled and configured correctly. It also brings in new features that differs from EBZip such as, | ||
+ | * The ability to use libdeflate algorithm at compile time, which, "is a library for fast, whole-buffer DEFLATE-based compression and decompression."<ref>[https://github.com/ebiggers/libdeflate?tab=readme-ov-file#overview README.md Overview section - libdeflate - GitHub]</ref>, which was added on 2019-06-30.<ref>[http://green.ribbon.to/~ikazuhiro/dic/ebu.html EB Library with UTF-8 extension]</ref>. Additionally, the new maintainer provided benchmark results against uncompressed, zlib and libdeflate.<ref>[http://ikazuhiro.s206.xrea.com/article.php/20190701203038572 Compressing EPWING (using libdeflate)]</ref> | ||
+ | ** libdeflate can also decompress EPWING e-books compressed with zlib, as well as recompressing them with potentially higher compression ratio. This could subsequently make it a viable candidate instead of needing to alternate between EBZip and EBUZip, see the [[Talk:EBZip#The_viability_ebuzip_as_a_possible_drop-in_replacement_candidate|Talk:EBZip section on this]] which is covered in detail. | ||
+ | * Fixes to work on a newer codebase, as opposed to maintaining a series of patches on top of libeb.<ref>[https://sources.debian.org/data/main/e/eb/4.4.3-14/debian/patches/010_debian.patch 010_debian.patch - eb-4.4.3-14 - sources.debian.org]</ref> | ||
+ | |||
+ | Mr. Kazuhiro have also previously contributed to Mr. Kasahara's EBZip project, on 2000-06-24.<ref>[https://github.com/mistydemeo/eb/blob/master/ChangeLog.1#L1371 ChangeLog.1 (line 1371) - EB Library - Misty De Méo]</ref>. | ||
== Identification == | == Identification == | ||
− | A <code>EBZ</code> file begins with bytes {{magic|45 42 5a 69 70 15 00 00 00 00 | + | A <code>EBZ</code> file begins with bytes {{magic|45 42 5a 69 70 15 00 00 00 00}}, in ASCII this would be <code>"EBZip...."</code>. |
== Examples == | == Examples == | ||
− | '''These must be executed in the top level of (book) directory that contains <code>CATALOGS</code> file, otherwise it will default to your current directory <code>$(pwd)</code>.''' See [[ | + | '''These must be executed in the top level of (book) directory that contains <code>CATALOGS</code> file, otherwise it will default to your current directory <code>$(pwd)</code>.''' See [[EPWING#Content structure]] for the directory structure of book layout. |
+ | If you are using EBUZip, the name of the compiled binary is <code>ebuzip</code>, as opposed to <code>ebzip</code>. | ||
Show information about the file: | Show information about the file: | ||
ebzip --information | ebzip --information | ||
Decompress files: | Decompress files: | ||
− | ebzip --uncompress wikipedia- | + | ebzip --uncompress wikipedia-fpw |
Compress files: | Compress files: | ||
− | ebzip wikipedia- | + | ebzip wikipedia-fpw |
Test files: | Test files: | ||
− | ebzip --test wikipedia- | + | ebzip --test wikipedia-fpw |
Show help: | Show help: | ||
ebzip --help | ebzip --help |
Latest revision as of 16:07, 5 September 2025
Electronic Book Zip or EBZip is a compression format used in electronic dictionaries that were distributed primarily on compact discs (CD). It is also one of the many frontend utilities that uses the EB library for manipulating EPWING format.
The electronic dictionaries stored on computers eliminates the need to have physical media, and allows remote referencing if it was connected to a network, using the likes of dserver
or ebnetd
for instance. It is likely that compression was needed due to the fact that hard disks were expensive compared to compact discs, when having a number of electronic dictionaries stored on a computer, it made little sense to retain physical media just to reference a number of words every so often, and to juggle around ejecting/inserting discs into the computer. For the expense of sacrificing few CPU cycles and potentially insignificant amount of memory, these electronic dictionaries could be made to take less storage space[2], and allows faster dictionary lookups due to hard disks allowing random access.
The compression is lossless, initially using a modified zlib[3], then likely unmodified zlib by the original ebzip
author[4]. Do keep in mind that not everything is compressed, EBZ format is not a container like ZIP for instance, selected files (typically CATALOGS
and HONMON
) are compressed and have their filenames bearing these extensions, e.g. HONMON.ebz
.
Contents |
[edit] History
EB-like software for Unix-like began circulating on Japanese Usenet as early as September 1991 in the form of shar by Shingo Nishioka of Osaka University.[5][6] Versions before 2.1beta of dserver
lacked compression, which was introduced as early as 1996-03-25 by Yura Shunsuke.[7] Gradually over time, Motoyuki Kasahara, a member of Software Research Associates (SRA) and the author of EB Library, began developing ndtpd
which handles electronic dictionary or dictionaries over the network, replacing dserver
.[8][9][10] The first release of such software was around 1997-06-04 as ndtpd-1.0beta0
(via FTP) with which it included eb
which was a library. Earliest release on fj.sources was 1997-08-07 as ndtpd-1.0
, which also describes the relationship between dserver
and ndtpd
, as shar.[11][12][13][14][15][16][17][18][19][20] Until the release of ndtpd-3.0alpha0
(2000-06-04), eb
has always been included. EB Library began as a separate project with the first release in 1999-06-16 as eb-2.2
. Both of these projects continued until the discontinuation of ndtpd
(as a release) on 2003-05-24 with the version 3.1.5. eb
remained to be maintained until 2010-03-08 with the version 4.4.3.
[edit] EBUZip
EBUZip is a fork of EBZip that is maintained by Ito Kazuhiro. The 'U' in EBUZip denotes the ability to support JIS X 4081 UTF-8 extensions provided that it is compiled and configured correctly. It also brings in new features that differs from EBZip such as,
- The ability to use libdeflate algorithm at compile time, which, "is a library for fast, whole-buffer DEFLATE-based compression and decompression."[21], which was added on 2019-06-30.[22]. Additionally, the new maintainer provided benchmark results against uncompressed, zlib and libdeflate.[23]
- libdeflate can also decompress EPWING e-books compressed with zlib, as well as recompressing them with potentially higher compression ratio. This could subsequently make it a viable candidate instead of needing to alternate between EBZip and EBUZip, see the Talk:EBZip section on this which is covered in detail.
- Fixes to work on a newer codebase, as opposed to maintaining a series of patches on top of libeb.[24]
Mr. Kazuhiro have also previously contributed to Mr. Kasahara's EBZip project, on 2000-06-24.[25].
[edit] Identification
A EBZ
file begins with bytes 45 42 5a 69 70 15 00 00 00 00
, in ASCII this would be "EBZip...."
.
[edit] Examples
These must be executed in the top level of (book) directory that contains CATALOGS
file, otherwise it will default to your current directory $(pwd)
. See EPWING#Content structure for the directory structure of book layout.
If you are using EBUZip, the name of the compiled binary is ebuzip
, as opposed to ebzip
.
Show information about the file:
ebzip --information
Decompress files:
ebzip --uncompress wikipedia-fpw
Compress files:
ebzip wikipedia-fpw
Test files:
ebzip --test wikipedia-fpw
Show help:
ebzip --help
[edit] Sample file
Filename | Main server download link | Mirror server download link | Notes |
kanjidic_en.fpwebz.tar | EDRDG.org | USF.edu | Use 7-Zip to open on Windows machines for instance, otherwise Tar should suffice. |
[edit] Software (mostly Japanese only)
See also: Wikipedia:ja:EPWING#検索ソフトウェア
- Windows
- Linux
[edit] Links (mostly Japanese only)
- Wikipedia:ja:電子ブック (規格)
- Internet Archive's mirror of EB Library that was hosted on SRA's website
- EBU
[edit] References (mostly Japanese only)
- ↑ ChangeLog - SRA website mirrored on Internet Archive
- ↑ Kenkyusha dictionary CD and EPWING - sci.lang.japan NARKIVE - Internet Archive copy
- ↑ Last version released with internal zlib - README.altered - version 4.1.3 - Misty De Méo
- ↑ Internal zlib removed - Commit 742ddab (4.2) - Misty De Méo
- ↑ dserver/Electronic Book Dictionary server & clients (1/2) - AncientFJ - katsu.watanabe.name
- ↑ dserver/Electronic Book Dictionary server & clients (2/2) - AncientFJ - katsu.watanabe.name
- ↑ dserver 2.1-beta patch for compressed dictionary - AncientFJ - katsu.watanabe.name
- ↑ BookView 1.0 - AncientFJ - katsu.watanabe.name
- ↑ BookView 1.0.2 (Re: BookView 1.0) - AncientFJ - katsu.watanabe.name
- ↑ BookView 1.1 (1/3) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (0/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (1/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (2/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (3/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (4/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (5/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (6/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (7/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (8/9) - AncientFJ - katsu.watanabe.name
- ↑ ndtpd 1.0 (9/9) - AncientFJ - katsu.watanabe.name
- ↑ README.md Overview section - libdeflate - GitHub
- ↑ EB Library with UTF-8 extension
- ↑ Compressing EPWING (using libdeflate)
- ↑ 010_debian.patch - eb-4.4.3-14 - sources.debian.org
- ↑ ChangeLog.1 (line 1371) - EB Library - Misty De Méo