Byte Order Mark

From Just Solve the File Format Problem
Revision as of 22:01, 4 March 2016 by Jsummers (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
File Format
Name Byte Order Mark
Ontology

A Byte Order Mark (BOM) is a strategically-placed U+FEFF (ZERO WIDTH NO-BREAK SPACE) character at the beginning of a Unicode text file, or other block of Unicode text.

Contents

Discussion

There are two main schools of thought as to its purpose:

  1. Its purpose is to identify the endianness of a file whose encoding is otherwise already known (particularly useful with UTF-16).
  2. Its purpose is more general: to help computer programs guess the encoding of a file, even if they have no external information about what its encoding might be. Thus, the term "byte order mark" is something of a misnomer.

The idea of a BOM is undeniably a hack, but its benefits sometimes outweigh its drawbacks.

To make false positives less likely, the U+FFFE code point is permanently reserved, and will never be a meaningful code point.

Other usage of the U+FEFF character is deprecated, and U+2060 WORD JOINER is suggested instead.

Byte patterns of common BOMs

A file beginning with bytes 0xFE 0xFF is probably encoded in UTF-16 with big-endian byte order.

0xFF 0xFE suggests UTF-16 with little-endian byte order.

0xEF 0xBB 0xBF suggests UTF-8.

UTF-8

Whether UTF-8 files should ever use a BOM is a contentious issue. A good case can be made for either side of the argument. But note that if you need to read files written by third-party applications, that ship has sailed: existing UTF-8 files often do use a BOM.

External links

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox