Re: relationship between locales, stream encodings, and wchar_t
p7eregex@gmail.com wrote:
I'm writing a C++ library that deals with strings and streams.
wchar_t is my normal choice of character type, but I don't want to
enforce that on 8-bit char systems. What is the STL/TR1 way of taking
encoded streams (such as UTF-8) and reading them into wchar_t or char
and then using locales so that tolower() and toupper() work properly
for that locale.
One fact up front: the whole locales assume that one internally used
element, be it char or wchar_t, is exactly one character. That means that
it doesn't allow handling a char string that actually uses UTF-8 as
encoding. It further doesn't lend itself to using a wchar_t string with
UTF-16 as encoding.
Actually, I don't see a good way to make tolower() work anywhere reliably
and portably. OTOH, I have never missed that feature either, since I don't
do any text manipulation except concatenation or splitting on very simple
separators. If I had to, I would actually much rather take a look at ICU,
which has additional features for collation etc.
Other than that, there are codecvt<> facets in the locale which allow a
conversion between externally used bytes (represented as 'char') and
internally used ones (using 'char' or 'wchar_t').
[...] for case insensitive operations.
I want to make my library as flexible as possible, and not assume that
my strings are unicode values.
If you don't want to assume anything, how are you then planning to perform
case insensitive operations? This already requires some interpretation of
the input. Of course, what you could do is require the user to supply
functions that convert a string to either case or do case-insensitive
operations.
Uli
--
Sator Laser GmbH
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]