Path: ...!weretis.net!feeder9.news.weretis.net!panix!.POSTED.spitfire.i.gajendra.net!not-for-mail
From: cross@spitfire.i.gajendra.net (Dan Cross)
Newsgroups: comp.arch
Subject: Re: MSI interrupts
Date: Thu, 27 Mar 2025 12:46:06 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <vs3hae$ahk$1@reader1.panix.com>
References: <vqto79$335c6$1@dont-email.me> <3061a58cdec38c3aa6093cbb74ea004a@www.novabbs.org> <5n0FP.78185$qx73.45402@fx01.iad> <efd7a61f2d4995d42225ecf999b6786f@www.novabbs.org>
Injection-Date: Thu, 27 Mar 2025 12:46:06 -0000 (UTC)
Injection-Info: reader1.panix.com; posting-host="spitfire.i.gajendra.net:166.84.136.80";
	logging-data="10804"; mail-complaints-to="abuse@panix.com"
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: cross@spitfire.i.gajendra.net (Dan Cross)
Bytes: 3162
Lines: 48

In article <efd7a61f2d4995d42225ecf999b6786f@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:
>On Wed, 26 Mar 2025 23:48:17 +0000, Scott Lurndal wrote:
>> mitchalsup@aol.com (MitchAlsup1) writes:
>>>On Wed, 26 Mar 2025 21:35:01 +0000, Scott Lurndal wrote:
>>>> [snip]
>>>> If you can NAK a snoop, are you then assuming that all DMA is
>>>> non-coherent?   That will really piss off the kernel folks.
>>>
>>>Not at all. DMA is as coherent as the I/O MMU's translating
>>>PTE prescribes.
>>>
>>>But; why would any sane coder be performing an ATOMIC event on
>>>memory that can be blindly overwritten by DMA ??? Or for that
>>>mater, critical section data being overwritten (at some unknown
>>>and unpredictable time) by DMA ??
>>
>> Polling on a completion status which the device DMAs
>> (often with an allocate hint) is not uncommon.   A 32-bit
>> word in memory updated by the device - the allocate hint
>> might update the LLC directly and invalidate L1/L2 or
>> may update the target cache line that the driver is
>> polling directly.
>
>Why do this ATOMICally ?? why not just read it ?? Or is this a
>grab (read and modify so someone else doesn't get it.) ??

Consider multiword DMA descriptors in a ring buffer in DRAM,
shared with a device.  Manipulating these races against hardware
that's consuming the ring, and on many devices you need to do a
multiword write to transfer ownership of a descriptor to the
device for processing (say, setting both a DMA address and an
ownership bit in two consecutive words).  A few particular
models of ethernet controllers in Cortex-M7 series parts from ST
come to mind as examples of devices that do this.  Of course
proper maintenance of ring tail pointers in the device coupled
with a barrier can mitigate this, but it would be nice to do
both the descriptor and tail pointer updates atomically; instead
of maintaining a separate tail pointer into the ring, you could
just initiate a start operation by writing an arbitrary value
into a register on the device.

Indeed, for the aforementioned ST part, many drivers have gotten
this stuff wrong.  See, for example,
https://github.com/oxidecomputer/hubris/pull/1493 and
https://github.com/embassy-rs/embassy/pull/2155

	- Dan C.