UTF-16 - Revision history

Jsummers at 12:56, 6 November 2025

2025-11-06T12:56:15Z

Dan Tobias at 02:37, 21 May 2019

2019-05-21T02:37:04Z

Dan Tobias at 19:53, 19 May 2019

2019-05-19T19:53:14Z

Dan Tobias at 19:49, 19 May 2019

2019-05-19T19:49:36Z

Dan Tobias at 19:49, 19 May 2019

2019-05-19T19:49:13Z

Jsummers at 22:25, 4 March 2016

2016-03-04T22:25:35Z

Jsummers: /* Relationship with UCS-2 */

2013-09-24T23:45:51Z

‎Relationship with UCS-2

Jsummers: /* Byte-oriented encodings */

2013-09-24T23:44:06Z

‎Byte-oriented encodings

Jsummers at 00:24, 12 September 2013

2013-09-12T00:24:14Z

Jsummers: /* Format */

2013-09-11T01:27:33Z

‎Format

@@ Line 9: / Line 9: @@
 '''UCS Transformation Format—16-bit''' ('''UTF-16''') is a 16-bit [[Unicode]] [[Character Encodings|character encoding]]. It encodes a sequence of Unicode code points in a sequence of unsigned 16-bit integers (which we'll call ''code units'').
-Code points up to U+FFFF use one code unit, while higher code points use two. UTF-16 is not capable of encoding code points in the reserved range U+D800 to U+DFFF, but that range is not required for full Unicode support.
+Code points up to U+FFFF use one code unit, while higher code points use two. UTF-16 supports the full range of the modern Unicode standard: code points U+0000 to U+D7FF, and U+E000 to U+10FFFF.
 == Format ==

@@ Line 2: / Line 2: @@
 |formattype=electronic
 |subcat=Character encoding
 |charset=UTF-16
 |charsetaliases=csUTF16

@@ Line 20: / Line 20: @@
 == Byte-oriented encodings ==
-There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes. IANA has assigned the charset identifiers UTF16BE (MIBenum 1013) and  UTF16LE (MIBenum 1014) to these variants.
+There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes. IANA has assigned the charset identifiers UTF-16BE (MIBenum 1013) and  UTF-16LE (MIBenum 1014) to these variants.
 The code units are encoded according to the desired byte order, and the order of the code units is not changed. This means that for UTF-16LE, 4-byte code points use sort of a ''mixed-endian'' byte order.

@@ Line 20: / Line 20: @@
 == Byte-oriented encodings ==
-There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes. ICANN has assigned the charset identifiers UTF16BE (MIBenum 1013) and  UTF16LE (MIBenum 1014) to these variants.
+There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes. IANA has assigned the charset identifiers UTF16BE (MIBenum 1013) and  UTF16LE (MIBenum 1014) to these variants.
 The code units are encoded according to the desired byte order, and the order of the code units is not changed. This means that for UTF-16LE, 4-byte code points use sort of a ''mixed-endian'' byte order.

@@ Line 2: / Line 2: @@
 |formattype=electronic
 |subcat=Character encoding
 }}
 '''UCS Transformation Format—16-bit''' ('''UTF-16''') is a 16-bit [[Unicode]] [[Character Encodings|character encoding]]. It encodes a sequence of Unicode code points in a sequence of unsigned 16-bit integers (which we'll call ''code units'').
@@ Line 17: / Line 20: @@
 == Byte-oriented encodings ==
-There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes.
+There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes. ICANN has assigned the charset identifiers UTF16BE (MIBenum 1013) and  UTF16LE (MIBenum 1014) to these variants.
 The code units are encoded according to the desired byte order, and the order of the code units is not changed. This means that for UTF-16LE, 4-byte code points use sort of a ''mixed-endian'' byte order.

@@ Line 25: / Line 25: @@
 == Relationship with UCS-2 ==
-UTF-16 is compatible with [[UCS-2]], in the sense that a UTF-16 decoder can correctly decode UCS-2, provided that the UCS-2 data does not use any reserved Unicode code points. The reverse is not true: a UCS-2 decoder cannot correctly decode UTF-16.
+UTF-16 is compatible with [[UCS-2]], in the sense that a UTF-16 decoder can correctly decode UCS-2, provided that the UCS-2 data does not use any reserved Unicode code points. The reverse is not true: a UCS-2 decoder cannot necessarily decode UTF-16.
 == Specifications ==

@@ Line 19: / Line 19: @@
 There are two byte-oriented flavors of UTF-16: UTF-16BE (big-[[Endianness|endian]]) and UTF-16LE (little-endian). Each code unit uses 2 bytes, so each code ''point'' is encoded using either 2 or 4 bytes.
-The code units are encoded according to the desired byte order, and the order of the code units is not changed. (This means that for UTF-16LE, 4-byte code points use sort of a ''mixed-endian'' byte order.)
+The code units are encoded according to the desired byte order, and the order of the code units is not changed. This means that for UTF-16LE, 4-byte code points use sort of a ''mixed-endian'' byte order.
-Files encoded in UTF-16BE or UTF-16LE often begin with a [[Byte Order Mark]] (BOM) to distinguish them from each other, and to help distinguish them from other character encodings. However, the Unicode standard says<ref>[http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf Unicode 6.2, §2.6]</ref> that a file with a BOM should not be labeled with a charset of "UTF-16BE" or "UTF-16LE". Instead, the charset name should be simply "UTF-16".
+Files encoded in UTF-16 often begin with a [[Byte Order Mark]] (BOM) to indicate their byte order, and to help distinguish them from other character encodings. But we need to be careful with terminology here, because the Unicode standard has decreed<ref>[http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf Unicode 6.2, §2.6]</ref> that the term "UTF-16BE" does ''not'' mean the same thing as "UTF-16 with big-endian byte order" (and similarly for UTF-16LE). A file encoded in "UTF-16" may use a BOM, but a file labeled as "UTF-16BE" or "UTF-16LE" should not use a BOM.
 == Relationship with UCS-2 ==

@@ Line 13: / Line 13: @@
 For higher code points, first subtract 0x10000 to get a 20-bit number. Add the 10 most significant bits of that number to 0xd800, and use that as the first code unit. Add the 10 least significant bits to 0xdc00, and use that as the second code unit.
-Descriptions of UTF-16 often use the term ''surrogate pair''. This tends to create unnecessary confusion, perhaps because whether a ''surrogate'' is a ''codepoint'' or not depends on context in subtle ways. See [[UCS-2#Surrogate pairs|UCS-2]] for the historical reasons that this term exists.
+Descriptions of UTF-16 often use the term ''surrogate pair''. This tends to create unnecessary confusion, perhaps because whether a surrogate pair is one code point or two depends on context in subtle ways. See [[UCS-2#Surrogate pairs|UCS-2]] for the historical reasons that this term exists.
 == Byte-oriented encodings ==

← Older revision		Revision as of 01:27, 11 September 2013
Line 12:		Line 12:

	For higher code points, first subtract 0x10000 to get a 20-bit number. Add the 10 most significant bits of that number to 0xd800, and use that as the first code unit. Add the 10 least significant bits to 0xdc00, and use that as the second code unit.		For higher code points, first subtract 0x10000 to get a 20-bit number. Add the 10 most significant bits of that number to 0xd800, and use that as the first code unit. Add the 10 least significant bits to 0xdc00, and use that as the second code unit.
		+
		+	Descriptions of UTF-16 often use the term ''surrogate pair''. This tends to create unnecessary confusion, perhaps because whether a ''surrogate'' is a ''codepoint'' or not depends on context in subtle ways. See [[UCS-2#Surrogate pairs\|UCS-2]] for the historical reasons that this term exists.

	== Byte-oriented encodings ==		== Byte-oriented encodings ==