Path: ...!weretis.net!feeder9.news.weretis.net!panix!.POSTED.spitfire.i.gajendra.net!not-for-mail
From: cross@spitfire.i.gajendra.net (Dan Cross)
Newsgroups: comp.unix.programmer
Subject: Re: OT: Windows (Was: Re: Open Source does not mean easily
Date: Tue, 7 Jan 2025 13:59:27 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <vljbvv$gl9$1@reader2.panix.com>
References: <uu54la$3su5b$6@dont-email.me> <vlgud7$1mgh5$1@dont-email.me> <vlh5ag$1nruu$1@dont-email.me> <677c7a1b$0$28501$426a74cc@news.free.fr>
Injection-Date: Tue, 7 Jan 2025 13:59:27 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="spitfire.i.gajendra.net:166.84.136.80";
	logging-data="17065"; mail-complaints-to="abuse@panix.com"
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: cross@spitfire.i.gajendra.net (Dan Cross)
Bytes: 4460
Lines: 73

In article <677c7a1b$0$28501$426a74cc@news.free.fr>,
Nicolas George  <nicolas$george@salle-s.org> wrote:
>Kalevi Kolttonen, dans le message <vlh5ag$1nruu$1@dont-email.me>, a
> �crit�:
>> I am no expert, but I guess if you need to do async programming
>> on UNIX/Linux userspace, your best is to use POSIX Threads.
>
>Very common misconception.

This is correct. Threads are a concurrency mechanism.  You use
them to write (more or less) sequential execution flows that run
concurrently with respect to one another.  They are not needed
to do "async programming UNIX/Linux userspace".  Indeed, in some
sense they're sort of the antithesis of async programming.

>The communication mechanisms between POSIX
>threads and Unix I/O are completely alien to each-other: it is not possible
>to poll() on a thread condition, nor is it it possible to set up a condition
>to be woken by data on a file descriptor. As a result, anybody who tries to
>use threads to solve problems of I/O concurrency ends up having to implement
>a poll() or equivalent loop in each thread, defeating the purpose.

This, however, does not follow.  I don't see why "poll" is
strictly required for IO concurrency.

In addition to signalfd etc, there is no reason you cannot, say,
have a signal handler that broadcasts on a condition variable
after an asynchronous IO operation completes, thus waking up a
thread.

More generally, in the context of a multithreaded program,
(assuming the POSIX threading model) the "condition to be woken
by data on a file descriptor" is that the IO operation on which
the thread was blocked has completed or been interrupted.  In a
system with kernel scheduled threads, this simply means that the
thread becomes runnable again, and will be resumed at some
point; this is exactly analogous to the sort of state
transitions a process goes through in a traditional Unix-style
kernel when blocked on IO.

Moreover, nothing prevents a program from, say, sharing a
producer/consumer queue between threads where one extracts
entries and performs IO on them while the other prepares IO
operations to be performed by the worker.  For that matter, one
can have multiple IO threads pushing work into a queue that's
executed by a pool of worker threads.

>POSIX threads are good to improve on computation concurrency, but they do
>not make I/O concurrency simpler, quite the opposite.
>
>The same might not be true for other kinds of threads.

A nice property of POSIX threads is that, in _most_
implementations, if you go off and initiate some blocking
operation, such as performing (blocking) IO on a file descriptor
of some sort, other runnable threads in the system can continue
executing.  Thus, it follows that you can have multiple IO
operations in separate threads pending at one time.  In that
sense, one can leverage threads for IO concurrency (indeed, many
programs do just that).  Threads can block independently without
impeding execution of the overall program.

Of course, not all POSIX thread implementations are capable of
doing this: consider a purely userspace "green" thread
implementation; executing a single blocking system call blocks
the whole program.  In those, in many cases, blocking IO is
emulated at the library level (which translates "normal" IO
operations into their non-blocking counterparts) to give the
illusion of sequential program flow; of course, the simulacrum
is inexact at best, which is why most systems push thread
management up into the kernel.

	- Dan C.