MySQL 8.0 Release Notes
MySQL 8.0 Source Code Documentation
The utf16 character set is the
ucs2 character set with an extension that
enables encoding of supplementary characters:
For a BMP character,
utf16anducs2have identical storage characteristics: same code values, same encoding, same length.For a supplementary character,
utf16has a special sequence for representing the character using 32 bits. This is called the “surrogate” mechanism: For a number greater than0xffff, take 10 bits and add them to0xd800and put them in the first 16-bit word, take 10 more bits and add them to0xdc00and put them in the next 16-bit word. Consequently, all supplementary characters require 32 bits, where the first 16 bits are a number between0xd800and0xdbff, and the last 16 bits are a number between0xdc00and0xdfff. Examples are in Section 15.5 Surrogates Area of the Unicode 4.0 document.
Because utf16 supports surrogates and
ucs2 does not, there is a validity check that
applies only in utf16: You cannot insert a
top surrogate without a bottom surrogate, or vice versa. For
example:
INSERT INTO t (ucs2_column) VALUES (0xd800); /* legal */
INSERT INTO t (utf16_column)VALUES (0xd800); /* illegal */
There is no validity check for characters that are technically
valid but are not true Unicode (that is, characters that Unicode
considers to be “unassigned code points” or
“private use” characters or even
“illegals” like 0xffff). For
example, since U+F8FF is the Apple Logo, this
is legal:
INSERT INTO t (utf16_column)VALUES (0xf8ff); /* legal */Such characters cannot be expected to mean the same thing to everyone.
Because MySQL must allow for the worst case (that one character
requires four bytes) the maximum length of a
utf16 column or index is only half of the
maximum length for a ucs2 column or index.
For example, the maximum length of a MEMORY
table index key is 3072 bytes, so these statements create tables
with the longest permitted indexes for ucs2
and utf16 columns:
CREATE TABLE tf (s1 VARCHAR(1536) CHARACTER SET ucs2) ENGINE=MEMORY;
CREATE INDEX i ON tf (s1);
CREATE TABLE tg (s1 VARCHAR(768) CHARACTER SET utf16) ENGINE=MEMORY;
CREATE INDEX i ON tg (s1);