Re: New utf8string design may make UTF-8 the superior encoding
On May 19, 1:21 pm, James Kanze <james.ka...@gmail.com> wrote:
On May 19, 12:01 am, =D6=F6 Tiib <oot...@hot.ee> wrote:
On 18 mai, 17:18, James Kanze <james.ka...@gmail.com> wrote:
[...]
But the trade-offs only concern internal representation.
Externally, the world is 8 bits, and UTF-8 is the only solution.
I would be honestly extremely glad if it was the only solution. Real
life applications throw in texts in all possible forms also they await
responses in all possible forms.
Yes. I meant it is the only solution if you are choosing
yourself. In practice, there are a lot of other solutions being
used; they don't work, except in limited environments, but they
are being widely used.
For example texts in financial transactions done in most
Northern Europe assume that "/\{}[]" means something like
"=C4=E4=C5=E5=D6=F6" (i do not remember correct order, but something li=
ke
that).
I prefer to convert incoming texts into std::wstring. Outgoing
texts i convert back to whatever they await (UTF-8 is really
relaxing news there, true). All what i need is a set of
conversion functions. If it is going to user interface then
std::wstring goes and it is business of UI to convert it
further into CString or QString or whatever they enjoy there
and sort it out for user.
In theory, the conversion should take place in the filebuf,
using the imbued locale.
Yes, if it is good wfilebuf then my problems are totally unexisting.
Often it is not in practice; instead there are strange protocol layers
and security by obscurity.
I perhaps have too low experience with sophisticated text processing.
Simple std::sort(), wide char literals of C++ and boost::wformat plus
full set of conversion functions is all i need really. Peter Olcott
raises lot of noise around it and so it makes me a bit
interested. :)
There can be advantages to using UTF-8 internally, as well as at
the interface level, and if you're not doing too complicated
things, it can work quite nicely. But only as long as your
manipulations aren't too complicated.
My major advantage from using wstring is that ...
Bytes are often too ambiguous information, even if exception like
UTF-8 the information is fully sufficient. Compiler does not make
difference between byte (char) in UTF-8 string, or byte in string in
some other encoding. wstring ensures that compilers/tools can easily
frown upon such bytes that sneak into application layer in whatever
encoding these are and from where-ever these come. That gains
attention at right place and for right reason.
For example there is:
basic_fstream::basic_fstream(const char* s, ios_base::openmode
mode);
If i give wstring::c_str() result as parameter s to that constructor i
get error. So compiler drags my attention to right place. If i get no
error then there is most likely extension to STL that most likely
works correctly. Giving result of string::c_str() (that contains
UTF-8) creates most likely garbage-filled file name.