| Age | Commit message (Collapse) | Author | Lines |
|---|
|
this is a POSIX requirement. we previously relied on the underlying fd
(or other backend) seek operation to produce the error, but since
linux lseek now supports other seek modes (SEEK_DATA and SEEK_HOLE)
which do not interact well with stdio buffering, this is insufficient.
instead, explicitly check whence before performing any operations.
|
|
both passing a null pointer to memcpy with length 0, and adding 0 to a
null pointer, are undefined. in some sense this is 'benign' UB, but
having it precludes use of tooling that strictly traps on UB. there
may be better ways to fix it, but conditioning the operations which
are intended to be no-ops in the k==0 case on k being nonzero is a
simple and safe solution.
|
|
POSIX places an obscure requirement on popen which is like a limited
version of close-on-exec:
"The popen() function shall ensure that any streams from previous
popen() calls that remain open in the parent process are closed in
the new child process."
if the POSIX-future 'e' mode flag is passed, producing a pipe FILE
with FD_CLOEXEC on the underlying pipe, this requirement is
automatically satisfied. however, for applications which use multiple
concurrent popen pipes but don't request close-on-exec, fd leaks from
earlier popen calls to later ones could produce deadlock situations
where processes are waiting for a pipe EOF that will never happen.
to fix this, iterate through all open FILEs and add close actions for
those obtained from popen. this requires holding a lock on the open
file list across the posix_spawn call so that additional popen FILEs
are not created after the list is traversed. note that it's still
possible for another popen call to start and create its pipe while the
lock is held, but such pipes are created with O_CLOEXEC and only drop
close-on-exec status (when 'e' flag is omitted) under control of the
lock.
|
|
the newly allocated FILE * has not yet leaked to the application and
is only visible to stdio internals until popen returns. since we do
not change any fields of the structure observed by libc internals,
only the pipe_pid member, locking is not necessary.
|
|
popen was special-casing the possibility (only possible when the
parent closed stdin and/or stdout) that the child's end of the pipe
was already on the final desired fd number, in which case there was no
way to get rid of its close-on-exec flag in the child. commit
6fc6ca1a323bc0b6b9e9cdc8fa72221ae18fe206 made this unnecessary by
implementing the POSIX-future requirement that dup2 file actions with
equal source and destination fd values remove the close-on-exec flag.
|
|
as the outcome of Austin Group tracker issue #62, future editions of
POSIX have dropped the requirement that fork be AS-safe. this allows
but does not require implementations to synchronize fork with internal
locks and give forked children of multithreaded parents a partly or
fully unrestricted execution environment where they can continue to
use the standard library (per POSIX, they can only portably use
AS-safe functions).
up until recently, taking this allowance did not seem desirable.
however, commit 8ed2bd8bfcb4ea6448afb55a941f4b5b2b0398c0 exposed the
extent to which applications and libraries are depending on the
ability to use malloc and other non-AS-safe interfaces in MT-forked
children, by converting latent very-low-probability catastrophic state
corruption into predictable deadlock. dealing with the fallout has
been a huge burden for users/distros.
while it looks like most of the non-portable usage in applications
could be fixed given sufficient effort, at least some of it seems to
occur in language runtimes which are exposing the ability to run
unrestricted code in the child as part of the contract with the
programmer. any attempt at fixing such contracts is not just a
technical problem but a social one, and is probably not tractable.
this patch extends the fork function to take locks for all libc
singletons in the parent, and release or reset those locks in the
child, so that when the underlying fork operation takes place, the
state protected by these locks is consistent and ready for the child
to use. locking is skipped in the case where the parent is
single-threaded so as not to interfere with legacy AS-safety property
of fork in single-threaded programs. lock order is mostly arbitrary,
but the malloc locks (including bump allocator in case it's used) must
be taken after the locks on any subsystems that might use malloc, and
non-AS-safe locks cannot be taken while the thread list lock is held,
imposing a requirement that it be taken last.
|
|
also fix the lack of declaration (and thus hidden visibility) in
__stdio_close's use of __aio_close.
|
|
now that struct winsize is available via sys/ioctl.h once again,
including termios.h is not needed.
|
|
these have been adopted for future issue of POSIX as the outcome of
Austin Group issue 1151, and are simply functions performing the roles
of the historical ioctls. since struct winsize is being standardized
along with them, its definition is moved to the appropriate header.
there is some chance this will break source files that expect struct
winsize to be defined by sys/ioctl.h without including termios.h. if
this happens, further changes will be needed to have sys/ioctl.h
expose it too.
|
|
vfscanf() may use the variable 'alloc' uninitialized when taking the
branch introduced by commit b287cd745c2243f8e5114331763a5a9813b5f6ee.
Spotted by clang.
|
|
apparently this function was intended at some point to be used by
strto* family as well, and thus was put in its own file; however, as
far as I can tell, it's only ever been used by vsscanf. move it to the
same file to reduce the number of source files and external symbols.
|
|
|
|
this idea came up when I thought we might need to zero the UNGET
portion of buf as well, but it seems like a useful improvement even
when that turned out not to be necessary.
|
|
as reported/analyzed by Pascal Cuoq, the shlim and shcnt
macros/functions are called by the scanf core (vfscanf) with f->rpos
potentially null (if the FILE is not yet activated for reading at the
time of the call). in this case, they compute differences between a
null pointer (f->rpos) and a non-null one (f->buf), resulting in
undefined behavior.
it's unlikely that any observably wrong behavior occurred in practice,
at least without LTO, due to limits on what's visible to the compiler
from translation unit boundaries, but this has not been checked.
fix is simply ensuring that the FILE is activated for read mode before
entering the main scanf loop, and erroring out early if it can't be.
|
|
this reverts commit 4ee039f3545976f9e3e25a7e5d7b58f1f2316dc3, which
added the helper as a hack to make vdprintf usable before relocation,
contingent on strong assumptions about the arch and tooling, back when
the dynamic linker did not have a real staged model for
self-relocation. since commit f3ddd173806fd5c60b3f034528ca24542aecc5b9
this has been unnecessary and the function was just wasting size and
execution time.
|
|
because struct stat is no longer assumed to correspond to the
structure used by the stat-family syscalls, it's not valid to make any
of these syscalls directly using a buffer of type struct stat.
commit 9493892021eac4edf1776d945bcdd3f7a96f6978 moved all logic around
this change for stat-family functions into fstatat.c, making the
others wrappers for it. but a few other direct uses of the syscall
were overlooked. the ones in tmpnam/tempnam are harmless since the
syscalls are just used to test for file existence. however, the uses
in fchmodat and __map_file depend on getting accurate file properties,
and these functions may actually have been broken one or more mips
variants due to removal of conversion hacks from syscall_arch.h.
as a low-risk fix, simply use struct kstat in place of struct stat in
the affected places.
|
|
aside from the special value EOF, ungetc is specified to accept and
convert values outside the range of unsigned char. conversion takes
place automatically as part of assignment when storing into the
buffer, but the return value is also required to be the resulting
converted value, and this requirement was not satisfied.
simplified from patch by Wang Jianjian.
|
|
commit cc3a4466605fe8dfc31f3b75779110ac93055bc1 fixed this for printf
but neglected to fix wprintf.
Previously, %lf caused a failure to output.
|
|
this probably saves a few bytes, avoids duplicating the clunky
lseek/_llseek syscall convention in two places, and sets the stage for
fixing broken seeks on x32 and mipsn32.
|
|
previously, POSIX erroneously required this to fail with EINVAL
despite the traditional glibc implementation, on which the POSIX
interface was based, allowing it. the resolution of Austin Group issue
818 removes the requirement to fail.
|
|
this is a requirement in POSIX that's omitted, and seemed potentially
non-conforming, in the C standard. as such it was omitted here.
however, as part of Austin Group issue #1170, the discrepancy was
raised with WG14 and determined to be unintended; future versions of
the C standard will require the error indicator to be set, as POSIX
does.
|
|
|
|
POSIX requires setvbuf to return non-zero if `mode` is not one of _IONBF,
_IOLBF, or _IOFBF.
|
|
the way gets was implemented in terms of fgets, it used the location
of the null termination to determine where to find and remove the
newline, if any. an embedded null byte prevented this from working.
this also fixes a one-byte buffer overflow, whereby when gets read an
N-byte line (not counting newline), it would store two null
terminators for a total of N+2 bytes. it's unlikely that anyone would
care that a function whose use is pretty much inherently a buffer
overflow writes too much, but it could break the only possible correct
uses of this function, in conjunction with input of known format from
a trusted/same-privilege-domain source, where the buffer length may
have been selected to exactly match a line length contract.
there seems to be no correct way to implement gets in terms of a
single call to fgets or scanf, and using multiple calls would require
explicit locking, so we might as well just write the logic out
explicitly character-at-a-time. this isn't fast, but nobody cares if a
catastrophically unsafe function that's so bad it was removed from the
C language is fast.
|
|
commit ddc947eda311331959c73dbc4491afcfe2326346 fixed the
corresponding bug for exit which was introduced when commit
0b80a7b0404b6e49b0b724e3e3fe0ed5af3b08ef added support for
caller-provided buffers, making it possible for stderr to be a
buffered stream.
|
|
fflush(NULL) and __stdio_exit lock individual FILEs while holding the
open file list lock to walk the list. since fclose first locked the
FILE to be closed, then the ofl lock, it could deadlock with these
functions.
also, because fclose removed the FILE to be closed from the open file
list before flushing and closing it, a concurrent fclose or exit could
complete successfully before fclose flushed the FILE it was closing,
resulting in data loss.
reorder the body of fclose to first flush and close the file, then
remove it from the open file list only after unlocking it. this
creates a window where consumers of the open file list can see dead
FILE objects, but in the absence of undefined behavior on the part of
the application, such objects will be in an inactive-buffer state and
processing them will have no side effects.
__unlist_locked_file is also moved so that it's performed only for
non-permanent files. this change is not necessary, but preserves
consistency (and thereby provides safety/hardening) in the case where
an application uses one of the standard streams after closing it while
holding an explicit lock on it. such usage is of course undefined
behavior.
|
|
check whether the lock is free before loading the calling thread's
tid. if so, just use a dummy tid value that cannot compare equal to
any actual thread id (because it's one bit wider). this also avoids
the need to save the tid and pass it to locking_getc or locking_putc,
reducing register pressure.
this change might slightly hurt the case where the caller already
holds the lock, but it does not affect the single-threaded case, and
may significantly improve the multi-threaded case, especially on archs
where loading the thread pointer is disproportionately expensive like
early mips and arm ISA levels. but even on i386 it helps, at least on
some machines; I measured roughly a 10-15% improvement.
|
|
commit d664061adb4d7f6647ab2059bc351daa394bf5da inadvertently omitted
the new file putc.h.
|
|
by ABI, the public stdin/out/err macros use extern pointer objects,
and this is necessary to avoid copy relocations that would be
expensive and make the size of the FILE structure part of the ABI.
however, internally it makes sense to access the underlying FILE
objects directly. this avoids both an indirection through the GOT to
find the address of the stdin/out/err pointer objects (which can't be
computed PC-relative because they may have been moved to the main
program by copy relocations) and an indirection through the resulting
pointer object.
in most places this is just a minor optimization, but in the case of
getchar and putchar (and the unlocked versions thereof), ipa constant
propagation makes all accesses to members of stdin/out PC-relative or
GOT-relative, possibly reducing register pressure as well.
|
|
this is the analog of commit dd8f02b7dce53d6b1c4282439f1636a2d63bee01,
but for putc.
|
|
with these changes, in a program that has not created any threads
besides the main thread and that has not called f[try]lockfile, getc
performs indistinguishably from getc_unlocked. this was measured on
several i386 and x86_64 models, and should hold on other archs too
simply by the properties of the code generation.
the case where the caller already holds the lock (via flockfile) is
improved significantly as well (40-60% reduction in time on machines
tested) and the case where locking is needed is improved somewhat
(roughly 10%).
the key technique used here is forcing the non-hot path out-of-line
and enabling it to be a tail call. a static noinline function
(conditional on __GNUC__) is used rather than the extern hiddens used
elsewhere for this purpose, so that the compiler can choose
non-default calling conventions, making it possible to tail-call to a
callee that takes more arguments than the caller on archs where
arguments are passed on the stack or must have space reserved on the
stack for spilling the. the tid could just be reloaded via the thread
pointer in locking_getc, but that would be ridiculously expensive on
some archs where thread pointer load requires a trap or syscall.
|
|
don't repeat definition in two places.
|
|
The condition occurs when
- thread #1 is holding the lock
- thread #2 is waiting for it on __futexwait
- thread #1 is about to release the lock and performs a_swap
- thread #3 enters the __lockfile function and manages to grab the lock
before thread #1 calls __wake, resetting the MAYBE_WAITERS flag
- thread #1 calls __wake
- thread #2 wakes up but goes again to __futexwait as the lock is
held by thread #3
- thread #3 releases the lock but does not call __wake as the
MAYBE_WAITERS flag is not set
This condition results in thread #2 not being woken up. This patch fixes
the problem by making the woken up thread ensure that the flag is
properly set before going to sleep again.
Mainainer's note: This fixes a regression introduced in commit
c21f750727515602a9e84f2a190ee8a0a2aeb2a1.
|
|
commit b114190b29417fff6f701eea3a3b3b6030338280 introduced spurious
realloc of the output buffer in cases where the result would exactly
fit in the caller-provided buffer. this is contrary to a strict
reading of the spec, which only allows realloc when the provided
buffer is "of insufficient size".
revert the adjustment of the realloc threshold, and instead push the
byte read by getc_unlocked (for which the adjustment was made) back
into the stdio buffer if it does not fit in the output buffer, to be
read in the next loop iteration.
in order not to leave a pushed-back byte in the stdio buffer if
realloc fails (which would violate the invariant that logical FILE
position and underlying open file description offset match for
unbuffered FILEs), the OOM code path must be changed. it would suffice
move just one byte in this case, but from a QoI perspective, in the
event of ENOMEM the entire output buffer (up to the allocated length
reported via *n) should contain bytes read from the FILE stream.
otherwise the caller has no way to distinguish trunated data from
uninitialized buffer space.
the SIZE_MAX/2 check is removed since the sum of disjoint object sizes
is assumed not to be able to overflow, leaving just one OOM code path.
|
|
morally, for null pointers a and b, a-b, a<b, and a>b should all be
defined as 0; however, C does not define any of them.
the stdio implementation makes heavy use of such pointer comparison
and subtraction for buffer logic, and also uses null pos/base/end
pointers to indicate that the FILE is not in the corresponding (read
or write) mode ready for accesses through the buffer.
all of the comparisons are fixed trivially by using != in place of the
relational operators, since the opposite relation (e.g. pos>end) is
logically impossible. the subtractions have been reviewed to check
that they are conditional the stream being in the appropriate reading-
or writing-through-buffer mode, with checks added where needed.
in fgets and getdelim, the checks added should improve performance for
unbuffered streams by avoiding a do-nothing call to memchr, and should
be negligible for buffered streams.
|
|
if EINVAL or ENOMEM happened before the first getc_unlocked, it was
possible that the stream orientation had not yet been set.
|
|
this further reduces the number of source files which need to include
libc.h and thereby be potentially exposed to libc global state and
internals.
this will also facilitate further improvements like adding an inline
fast-path, if we want to do so later.
|
|
the LFS64 macro was not self-documenting and barely saved any
characters. simply use weak_alias directly so that it's clear what's
being done, and doesn't depend on a header to provide a strange macro.
|
|
libc.h was intended to be a header for access to global libc state and
related interfaces, but ended up included all over the place because
it was the way to get the weak_alias macro. most of the inclusions
removed here are places where weak_alias was needed. a few were
recently introduced for hidden. some go all the way back to when
libc.h defined CANCELPT_BEGIN and _END, and all (wrongly implemented)
cancellation points had to include it.
remaining spurious users are mostly callers of the LOCK/UNLOCK macros
and files that use the LFS64 macro to define the awful *64 aliases.
in a few places, new inclusion of libc.h is added because several
internal headers no longer implicitly include libc.h.
declarations for __lockfile and __unlockfile are moved from libc.h to
stdio_impl.h so that the latter does not need libc.h. putting them in
libc.h made no sense at all, since the macros in stdio_impl.h are
needed to use them correctly anyway.
|
|
|
|
commits leading up to this one have moved the vast majority of
libc-internal interface declarations to appropriate internal headers,
allowing them to be type-checked and setting the stage to limit their
visibility. the ones that have not yet been moved are mostly
namespace-protected aliases for standard/public interfaces, which
exist to facilitate implementing plain C functions in terms of POSIX
functionality, or C or POSIX functionality in terms of extensions that
are not standardized. some don't quite fit this description, but are
"internally public" interfacs between subsystems of libc.
rather than create a number of newly-named headers to declare these
functions, and having to add explicit include directives for them to
every source file where they're needed, I have introduced a method of
wrapping the corresponding public headers.
parallel to the public headers in $(srcdir)/include, we now have
wrappers in $(srcdir)/src/include that come earlier in the include
path order. they include the public header they're wrapping, then add
declarations for namespace-protected versions of the same interfaces
and any "internally public" interfaces for the subsystem they
correspond to.
along these lines, the wrapper for features.h is now responsible for
the definition of the hidden, weak, and weak_alias macros. this means
source files will no longer need to include any special headers to
access these features.
over time, it is my expectation that the scope of what is "internally
public" will expand, reducing the number of source files which need to
include *_impl.h and related headers down to those which are actually
implementing the corresponding subsystems, not just using them.
|
|
this functions is glue for linking dependency logic.
|
|
logically these belong to the intersection of the stdio and pthread
subsystems, and either place the declarations could go (stdio_impl.h
or pthread_impl.h) requires a forward declaration for one of the
argument types.
|
|
policy is that all public functions which have a public declaration
should be defined in a context where that public declaration is
visible, to avoid preventable type mismatches.
an audit performed using GCC's -Wmissing-declarations turned up the
violations corrected here. in some cases the public header had not
been included; in others, a feature test macro needed to make the
declaration visible had been omitted.
in the case of gethostent and getnetent, the omission seems to have
been intentional, as a hack to admit a single stub definition for both
functions. this kind of hack is no longer acceptable; it's UB and
would not fly with LTO or advanced toolchains. the hack is undone to
make exposure of the declarations possible.
|
|
this requirement is specified by POSIX.
|
|
if no output is produced, no underlying fwrite will ever be called,
but byte-oriented printf functions are still required to set the
orientation of the stream to byte-oriented. call __towrite explicitly
if the FILE is not already in write mode.
|
|
commit b5a8b28915aad17b6f49ccacd6d3fef3890844d1 setup the write buffer
bound pointers for the temporary buffer manually to fix a buffer
overflow issue, but in doing so, caused vfprintf on unbuffered files
never to call __towrite, thereby failing to set the stream orientation
to byte-oriented, failing to clear any prior read mode, and failing to
produce an error when the stream is not writable.
revert the inline setup of the bounds pointers and instead zero them,
so that the underlying fwrite code will call __towrite to set them up.
|
|
commit 0b80a7b0404b6e49b0b724e3e3fe0ed5af3b08ef added the ability to
set application-provided stdio FILE buffers, adding the possibility
that stderr might be buffered at exit time, but __stdio_exit did not
have code to flush it.
this regression was not present in any release.
|
|
|
|
fundamentally there is no good reason these functions need to set an
orientation (morally it should be possible to write a wchar_t[] memory
stream using byte functions, or a char[] memory stream using wide
functions), but it's a part of the specification that they do. aside
from being able to inspect the orientation with fwide, failure to set
the orientation in open_wmemstream is observable if the locale changes
between open_wmemstream and the first operation on the stream; this is
because the encoding rule (locale) for the stream is required to be
bound at the time the stream becomes wide-oriented.
for open_wmemstream, call fwide to avoid duplicating the logic for
binding the encoding rule. for open_memstream it suffices just to set
the mode field in the FILE struct.
|
