| Age | Commit message (Collapse) | Author | Lines |
|---|
|
despite clarifications made to the COPYRIGHT file in commit
f0a61399330bae42beeb27d6ecd05570b3382a60, there continues to be
confusion about whether the permissions granted actually apply to all
files. I am the sole author of these files and clearly intend, and
have always intended, for the grant of permission to apply to them.
|
|
this patch makes the functions which work directly on multibyte
characters treat the high bytes as individual abstract code units
rather than as multibyte sequences when MB_CUR_MAX is 1. since
MB_CUR_MAX is presently defined as a constant 4, all of the new code
added is dead code, and optimizing compilers' code generation should
not be affected at all. a future commit will activate the new code.
as abstract code units, bytes 0x80 to 0xff are represented by wchar_t
values 0xdf80 to 0xdfff, at the end of the surrogates range. this
ensures that they will never be misinterpreted as Unicode characters,
and that all wctype functions return false for these "characters"
without needing locale-specific logic. a high range outside of Unicode
such as 0x7fffff80 to 0x7fffffff was also considered, but since C11's
char16_t also needs to be able to represent conversions of these
bytes, the surrogate range was the natural choice.
|
|
|
|
issue reported by Michael Forney:
"If wn becomes 0 after processing a chunk of 4, mbsrtowcs currently
continues on, wrapping wn around to -1, causing the rest of the string
to be processed.
This resulted in buffer overruns if there was only space in ws for wn
wide characters."
the original patch submitted added an additional check for !wn after
the loop; to avoid extra branching, I instead just changed the wn>=4
check to wn>=5 to ensure that at least one slot remains after the
word-at-a-time loop runs. this should not slow down the tail
processing on real-world usage, since an extra slot that can't be
processed in the word-at-a-time loop is needed for the null
termination anyway.
|
|
|
|
these changes fix at least two bugs:
- misaligned access to the input as uint32_t for vectorized ASCII test
- incorrect src pointer after stopping on EILSEQ
in addition, the text of the standard makes it unclear whether the
mbstate_t object is to be modified when the destination pointer is
null; previously it was cleared either way; now, it's only cleared
when the destination is non-null. this change may need revisiting, but
it should not affect most applications, since calling mbsrtowcs with
non-zero state can only happen when the head of the string was already
processed with mbrtowc.
finally, these changes shave about 20% size off the function and seem
to improve performance by 1-5%.
|
|
to deal with the fact that the public headers may be used with pre-c99
compilers, __restrict is used in place of restrict, and defined
appropriately for any supported compiler. we also avoid the form
[restrict] since older versions of gcc rejected it due to a bug in the
original c99 standard, and instead use the form *restrict.
|
|
this code was written independently of musl, with support for a the
backwards, nonstandard "31-bit unicode" some libraries/apps might
want. unfortunately the extra code (inside #ifdef) makes the source
harder to read and makes code that should be simple look complex, so
i'm removing it. anyone who wants to use the old code can find it in
the history or from elsewhere.
also, change the visibility of the __fsmu8 state machine table to
hidden, if supported. this should improve performance slightly in
shared-library builds.
|
|
|
|
|
