Path: news.eternal-september.org!eternal-september.org!feeder3.eternal-september.org!panix!.POSTED.spitfire.i.gajendra.net!not-for-mail
From: cross@spitfire.i.gajendra.net (Dan Cross)
Newsgroups: comp.arch
Subject: Re: MSI interrupts
Date: Sat, 29 Mar 2025 20:55:45 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <vs9moh$79f$1@reader1.panix.com>
References: <vqto79$335c6$1@dont-email.me> <8uzEP.115550$Xq5f.66883@fx38.iad> <vruq1p$cv2$1@reader1.panix.com> <t3CFP.529522$f81.217510@fx48.iad>
Injection-Date: Sat, 29 Mar 2025 20:55:45 -0000 (UTC)
Injection-Info: reader1.panix.com; posting-host="spitfire.i.gajendra.net:166.84.136.80";
	logging-data="7471"; mail-complaints-to="abuse@panix.com"
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: cross@spitfire.i.gajendra.net (Dan Cross)

In article <t3CFP.529522$f81.217510@fx48.iad>,
EricP  <ThatWouldBeTelling@thevillage.com> wrote:
>Dan Cross wrote:
>> In article <8uzEP.115550$Xq5f.66883@fx38.iad>,
>> EricP  <ThatWouldBeTelling@thevillage.com> wrote:
>>> Dan Cross wrote:
>>>> In article <kVgEP.1277108$_N6e.605199@fx17.iad>,
>>>> EricP  <ThatWouldBeTelling@thevillage.com> wrote:
>>>>> Dan Cross wrote:
>>>>>> [snip]
>>>>>> Consider a thread that takes a spinlock; suppose some
>>>>>> high-priority interrupt comes in while the thread is holding
>>>>>> that lock.  In response to the interrupt, software decides to
>>>>>> suspend the thread and switch some other thread; that thread
>>>>>> wants to lock the spin lock that the now-descheduled thread is
>>>>>> holding: a classic deadlock scenario.
>>>>> Terminology: mutexes coordinate mutual exclusion between threads,
>>>>> spinlocks coordinate mutual exclusion between cpu cores.
>>>>> Windows "critical sections" are mutexes with a fast path.
>>>> A spin lock is simply any lock where you spin trying to acquire
>>>> the lock, as opposed to a blocking synchronization protocol.
>>>> Here I'm using the terminology of Herlihy and Shavit [Her08].
>>> I'm using terminology in common use on multiprocessor systems like
>>> VMS since 1980's and later WinNT.
>>>
>>> Each OS data structure has different reentrancy requirements.
>>> - Ones accessed by threads are guarded by mutexes.
>>>   Example on Windows would be the page tables as paging is done by threads.
>>> - Ones accessed by OS software interrupt level are guarded by spinlocks.
>>>   Example on Windows is the Scheduler tables.
>>> - Ones accessed by HW interrupts are guarded by interrupt spinlocks.
>>>   Example on Windows is device interrupt service routines.
>> 
>> I'm not sure what you're saying here, or rather, how it is
>> relevant.
>> 
>> Your earlier statement was that "mutexes coordinate mutual
>> exclusion between threads, spinlocks coordinate mutual exclusion
>> between cpu cores."  But this is not the case.
>> 
>> Taking VMS as an example, from [Gol94], sec 9.5 (page 255):
>> 
>> |A thread of execution running on one processor acquires a
>> |spinlock to serialize access to the data with threads of
>> |execution running on other processors.  Before acquiring
>> |the spinlock, the thread of executions raises IPL to block
>> |accesses by other threads of execution running on the same
>> |processor.  The IPL value is determined by the spinlock being
>> |locked.
>> 
>> Goldberg and Saravanan seems quite explicit that threads can
>> acquire and hold spinlocks on OpenVMS, which seems to be
>> something that you were objecting to?
>
>It took me a while to find  copy of that book online.
>That 1994 book relates to VMS before they added kernel threads to the OS.

Before you said you were using terminology commonly used on
systems "since [the] 1980s".  The systems you cited were VMS and
Windows.  I quoted references about both systems from the mid
1990s and mid 2000s, respectively.  If that terminology were
common in the 1980s, as you suggested, surely it would be in a
book from 1994?  Regardless, see below.

>At that point VMS scheduled processes, an execution context plus virtual
>space, like most *nix (except I think Solaris which had kernel threads).
>
>It says on pg 3: "The term thread of execution as used in this book
>refers to a computational entity, an agent of execution."
>This refers generally to the various code sequences from different context
>that may be intertwined on a processors core for interrupts, exceptions,
>Dpc (called driver forks in VMS), process AST's (software interrupts),
>and applications.
>
>In a later book on VMS after they added kernel scheduled threads they
>consistently use the two terms "thread of execution" and "kernel thread"
>to distinguish the two meanings.

Are you referring to the 1997 revision of Goldenberg et al,
that covers VMS 7.0?  [Gol97]  In that context, VMS "kernel
threads" refer to threads of execution within a program that
are managed and scheduled by the VMS executive (aka kernel).

This is described on page 151, section 5.1:

|A thread of execution is a sequential flow of instruction
|executiuon.  A traditional program is single-threaded,
|consisting of one main thread of execution.  In contrast, a
|multithreaded program divides its work among multiple threads
|that can run concurrently.  OpenVMS supplies DECthreads, a
|run-time library, to facilitate the creation, synchronization,
|and deletion of multiple user-mode thredas.
|
|OpenVMS Alpha Version 7.0 supports multiple execution contexts
|within a process: each execution context has its own hardware
|context and stacks, and canexecute independently of the others;
|all execution context with a process share the same virtual
|address space.  The kernel _kernel thread_ refers to one of
|those execution contexts and the thread of execution running
|within it.  This volume uses the term _multithreaded_ to refer
|to a process with multiple kernel threads.
|
|The kernel thread is now the basis for OpenVMS scheduling.
|[...]

It continues on page 152:

|The term _kernel thread_ is a somewhat confusing name, in that
|instructions executing in a kernel thread can run in any of the
|four access modes, not only kernel mode.  The term arose to
|distinguish the current multithreaded model, supported within
|the OpenVMS executive (or kernel), from the previous model,
|supported only by DECthreads.

Sadly, the later books on OpenVMS internals are more focused on
specific subsystems (scheduling and process management; memory
management) than the ealier volumes.  Details of the
implementation of synchronization primitives are almost entirely
missing.

There is discussion about how those primitives are _used_,
however.  [Gol03] mentions spinlocks, such as the SCHED
spinlock, in the context of memory management data structures.
E.g., on page 51:

|The swappability of the PHD [Process Header] result in several
|different methods for synchronizing access to fields within it.
|Because a PHD can be inswapped to a different balance set slot
|than it last occuped, accesses to a PHD that use its system
|space address must be synchronized against swapper
|interference.  Accesses from a kernel thread to its own PHD can
|be made with the SCHED spinlock held to block any rescheduling
|and possible swapping of the process.  Holding the MMG spinlock
|is another way to block swapping.

Notably here, that the language is again in terms of (software)
threads.  It seems clear enough that in the VMS context,
spinlocks are conceptually held by threads, not cores.

>I find the phrase "thread of execution" ambiguous and try not use it.
>After the term thread had been pretty much adopted to mean kernel thread
>as an OS scheduled execution context, I would have preferred if they used
>something like "strand of execution" to avoid the overloading.

I disagree with that characterization.  "Thread" generically
means what they said it means above: a sequential flow of
instruction execution.  Generically, on most systems, "threads"
are just register state and a stack in some address address
space.

Fibers/Strands/etc tend to refer to lightweight cooperatively
scheduled concurrency structures.

Anyway, I don't think that much changes the point I was making.
I don't see any evidence of 

>> Turning to Windows, [Rus12] says about spin locks on page 179:
>> 
>> |Testing and acquiring the lock in one instruction prevents a
>> |second thread from grabbing the lock between the time the first
>> |thread tests the variable and the time it acquires the lock.
>> 
>> Note that, again, the language here is thread-centric.
>
>He says on pg  12: "A thread is the entity within a process that
>Windows schedules for execution." meaning "kernel thread".
>
>He also uses the term thread loosely as you can see on pg 13 when he says:
>"Fibers allow an application to schedule its own �threads� of execution"

I don't see how that changes the point.  

>He also says pg 87 (to paraphrase) that scheduling priority is an
>attribute of a thread whereas IRQL is an attribute of a processor.
>Which is the distinction I am trying to make: when you raise IRQL
>it stops being a thread executing and starts being a processor core.

The exact paragraph I suspect you are referring to is this:

|IRQL priority levels have a completely different meaning than
|thread-scheduling priorities (which are described in chapter
|5).  A scheduling priority is an attribute of a thread, whereas
|an IRQL is an attribute of an interrupt source, such as a
|keyboard or a mouse.  In addition, each processor has an IRQL
|setting that chnages as operating system code executes.

========== REMAINDER OF ARTICLE TRUNCATED ==========