Avoid the optimized mb_str() available in this build returning directly
a "char*" as it doesn't preserve the length of the string if it contains
NULs. Use mb_str(wxMBConv) overload instead which always returns the
buffer of the correct length.
Arguably, this is a problem of wxString API and maybe this optimized
mb_str() overload shouldn't be provided at all. OTOH the main reason for
wxUSE_UTF8_LOCALE_ONLY existence is optimization, so it seems a pity to
deliberately make the code less efficient when using it.
The current code incorrectly returned true if the string contained
an invalid UTF-8 sequence after an embedded NUL.
Check the entire string if the length was explicitly given instead of stopping
at the first NUL.
Closes https://github.com/wxWidgets/wxWidgets/pull/236
Don't optimize the required length as this is a tiny gain resulting in big
problems with the strings containing surrogates for which the actual result is
shorter than the length returned, resulting in extra NUL bytes at the end of
the converted buffer.
This is similar to 3410aa372f (see #16298) but
for UTF-32 and not UTF-16.
Closes#17070.
Valgrind complains about reading beyond the end of buffer when using glibc
std::string for wxString implementation under amd64 Linux. Don't suppose that
the buffer has one extra null word at its end to avoid this.
This test can't work when the in-memory representation is UTF-16, as we can't
convert surrogates to anything else in this case.
This fixes the unit tests broken since r76622, see #16298.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@78022 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
Don't stop converting subsequent chunks just because the length of one of them
was 0: this can happen if the first character of a string is a NUL or if there
are two (or more) NULs in it later.
Simply remove the check for this and continue as usual even in this case.
Also add a unit test verifying that we do translate NULs in input into NULs in
output.
Closes#16620.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@78021 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
Don't optimize the returned length for surrogate-less case, this does save a
pass of the string but at the price of returning a wrong result, which is not
worth it, just compute the really required length exactly.
Closes#16298.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@76622 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
We read beyond the provided maximal length as we didn't update the remaining
length while parsing the remaining bytes of an UTF-8-encoded code point.
Fix this and add a test for it.
Closes#15901.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@75733 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
This keyword is not expanded by Git which means it's not replaced with the
correct revision value in the releases made using git-based scripts and it's
confusing to have lines with unexpanded "$Id$" in the released files. As
expanding them with Git is not that simple (it could be done with git archive
and export-subst attribute) and there are not many benefits in having them in
the first place, just remove all these lines.
If nothing else, this will make an eventual transition to Git simpler.
Closes#14487.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@74602 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
Remove tests added in r64324, there is no need to have commented out code
here and UTF-8 test was pretty strange as well (it's far from clear what
should the result of converting "utf8Invalid" be).
This also suppresses a warning that these changes introduced.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@64387 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
When converting a fixed number of characters we need to take any NULs inside
the buffer being converted into account for our return value -- but this
wasn't done and converting 2 characters 'x' and '\0' returned only 1, even if
the length 2 was explicitly specified.
Fix this bug and add a unit test checking for it.
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@62141 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
All files in svn are supposed to use UTF-8 now so convert this one to UTF-8
too before committing more changes to it (which would be otherwise rejected by
pre-commit hook).
git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@62140 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
