libutils: Add more tests for Unicode
This CL added additional tests for converting between utf16 and utf8,
specifically tests that:
1. check utf16_to_utf8_length() returns 0 if input is an
empty UTF16 string
2. check utf16_to_utf8_length() returns 1 if input is a
single ASCII character UTF16 string
3. check utf16_to_utf8_length() returns 3 if input is a
single UTF-16 character between U+0800 - U+FFFF
4. check utf16_to_utf8_length() returns 4 if input has
a surrogate pair
5. check unpaired UTF-16 surrogate is handled correctly
(skipped)
6. check utf16_to_utf8_length(0 handles invalid surrogate
case correctly, by skipping the first but handling the
rest correctly
7. check a normal string with a mix of 1/2/3/4-byte UTF8
characters is correctly converted by utf16_to_utf8()
8. check conversion from invalid utf8 sequence with invalid
leading byte and/or invalid trailing byte(s) should still
work and not crash
Change-Id: If68e514af0e84ddebf5900b2e140e76ba4f44553
1 file changed