Path: ...!weretis.net!feeder9.news.weretis.net!i2pn.org!i2pn2.org!.POSTED!not-for-mail
From: mitchalsup@aol.com (MitchAlsup1)
Newsgroups: comp.arch
Subject: Re: PCIe MSI-X interrupts
Date: Wed, 10 Jul 2024 22:01:27 +0000
Organization: Rocksolid Light
Message-ID: <7147165f58d3afef17cf9e9e1dbbd2fd@www.novabbs.org>
References: <bb16865f7675526d4e2b87283e28c2c5@www.novabbs.org> <%LdfO.108407$xKj1.7795@fx09.iad> <ecd43e7ed4d3cc6fcc3bca3a999725e8@www.novabbs.org> <922220c8593353c7ed0fda9e656d359d@www.novabbs.org> <v6mjeq$20vo4$1@dont-email.me> <UUAjO.8178$BFg.3080@fx13.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
	logging-data="2842613"; mail-complaints-to="usenet@i2pn2.org";
	posting-account="65wTazMNTleAJDh/pRqmKE7ADni/0wesT78+pyiDW8A";
User-Agent: Rocksolid Light
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Rslight-Posting-User: ac58ceb75ea22753186dae54d967fed894c3dce8
X-Rslight-Site: $2y$10$s/eemOEeV3Hkn1GqsG0t9OVgAVXaRdxbxiVDpmZxVnvoDjnKQUGrC
Bytes: 3769
Lines: 67

On Wed, 10 Jul 2024 19:02:12 +0000, Scott Lurndal wrote:

> kegs@provalid.com (Kent Dickey) writes:
>>In article <922220c8593353c7ed0fda9e656d359d@www.novabbs.org>,
>>MitchAlsup1 <mitchalsup@aol.com> wrote:
>>>On page 43 of::
>>>https://www.cs.uml.edu/~bill/cs520/slides_15E_PCI_Express_IOV.pdf
>>>
>>>it states: "Must not indicate an invalidation has completed
>>>until all outstanding Read Requests that reference the
>>>associated translation have retired"
>>>
>>>"Must insure that the invalidation completion indication to RC
>>>will arrive at the RC after previously posted writes that use
>>>the stale address."
>>>
>>>and
>>>
>>>"...If transactions are in a queue waiting to be sent, It is
>>>not necessary for the device to expunge requests from the
>>>queue even if those transaction[s] use an address that is
>>>being invalidated."
>>>
>>>The first 2 seem to be PCIe ordering requirements between
>>>EP and RC.
>>>
>>>The 3rd seems to say if EP used a translation while it was
>>>valid, then its invalidation does not prevent requests
>>>using the now stale translation.
>>>
>>>So, a SATA device could receive a command to read a page
>>>into memory. SATA EP requests ATS for the translation of
>>>the given virtual address to the physical page. Then the
>>>EP creates a queue of write requests filling in the addr
>>>while waiting on data. Once said queue has been filled,
>>>and before the data comes off the disk, an invalidation
>>>arrives and is ACKed. The data is still allowed to write
>>>into memory.
>>>
>>>{{But any new command to the SATA device would not be
>>>allowed to use the translation.}}
>>>
>>>Is this a reasonable interpretation of that page?
>>
>>No, it's saying that the EP can keep using a stale translation UNTIL it
>>returns the ACK for an invalidation.  It does not need to toss those
>>requests--it just needs to delay the ACK.  Or it could toss the requests,
>>and then send the ACK faster, but it's optional if it wants to toss
>> requests.
>>
>
> Indeed.  And I'd suggest that the official PCI Express
> specification is a better source than a set of slides.
>
> From the spec:

I do not have access through the PCIe paywall.
>
> a. A Function is required not to indicate the invalidation has completed
> until
>    all outstanding Read Requests or Translation Requests that reference
> the
>    associated translated address have been retired or nullified.
> b. A Function is required to ensure that the Invalidate Completion
> indication
>    to the RC will arrive at the RC after any previously posted writes
> that use
>    the "stale" address.