Commit Graph

1 Commits (submit/fix-mingw-escaping)

Author SHA1 Message Date
Taylor R Campbell 212c85863a Avoid undefined behaviour with the ctype(3) functions.
fix https://github.com/pkgconf/pkgconf/issues/291

As defined in the C standard:

        In all cases the argument is an int, the value of which shall
        be representable as an unsigned char or shall equal the value
        of the macro EOF.  If the argument has any other value, the
        behavior is undefined.

This is because they're designed to work with the int values returned
by getc or fgetc; they need extra work to handle a char value.

If EOF is -1 (as it almost always is), with 8-bit bytes, the allowed
inputs to the ctype(3) functions are:

        {-1, 0, 1, 2, 3, ..., 255}.

However, on platforms where char is signed, such as x86 with the
usual ABI, code like

        char *ptr = ...;
        ... isspace(*ptr) ...

may pass in values in the range:

        {-128, -127, -126, ..., -2, -1, 0, 1, ..., 127}.

This has two problems:

1. Inputs in the set {-128, -127, -126, ..., -2} are forbidden.

2. The non-EOF byte 0xff is conflated with the value EOF = -1, so
   even though the input is not forbidden, it may give the wrong
   answer.

Casting char to unsigned int first before passing the result to
ctype(3) doesn't help: inputs like -128 are unchanged by this cast,
because (on a two's-complement machine with 32-bit int and unsigned
int), converting the signed char with integer value -128 to unsigned
int gives integer value 2^32 - 128 = 0xffffff80, which is out of
range, and which is converted in int back to -128, which is also out
of range.

It is necessary to cast char inputs to unsigned char first; you can
then cast to unsigned int if you like but there's no need because the
functions will always convert the argument to int by definition.  So
the above fragment needs to be:

        char *ptr = ...;
        ... isspace((unsigned char)*ptr) ...

This patch changes unsigned int casts to unsigned char casts, and
adds unsigned char casts where they are missing.
2023-05-02 11:43:56 -07:00