Path: ...!weretis.net!feeder9.news.weretis.net!news.nk.ca!rocksolid2!i2pn2.org!.POSTED!not-for-mail
From: mitchalsup@aol.com (MitchAlsup1)
Newsgroups: comp.arch
Subject: Re: MSI interrupts
Date: Thu, 13 Mar 2025 21:34:06 +0000
Organization: Rocksolid Light
Message-ID: <748c0cc0ba18704b4678fd553193573e@www.novabbs.org>
References: <vqto79$335c6$1@dont-email.me> <3d5200797dd507ae051195e0b2d8ff56@www.novabbs.org> <YyFAP.729659$eNx6.235106@fx14.iad> <6731f278e3a9eb70d34250f43b7f15f2@www.novabbs.org> <AUHAP.61125$Xq5f.14972@fx38.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
	logging-data="50533"; mail-complaints-to="usenet@i2pn2.org";
	posting-account="o5SwNDfMfYu6Mv4wwLiW6e/jbA93UAdzFodw5PEa6eU";
User-Agent: Rocksolid Light
X-Spam-Checker-Version: SpamAssassin 4.0.0
X-Rslight-Posting-User: cb29269328a20fe5719ed6a1c397e21f651bda71
X-Rslight-Site: $2y$10$lkF8N7gKoN4nP3Uh024TLuYJKIyVJLIIyutMlPF12NoEkiKX2MrD.
Bytes: 2905
Lines: 43

On Thu, 13 Mar 2025 21:14:08 +0000, Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:
>>On Thu, 13 Mar 2025 18:34:32 +0000, Scott Lurndal wrote:
>>
>
>>> Most modern devices advertise the MSI-X capability instead.
>>
>>And why not:: its just a few more flip-flops and almost no more
>>sequencing logic.
>
> I've seen several devices with more than 200 MSI-X vectors;
> thats 96 bits per vector to store a full 64-bit address and
> 32-bit data payload.

At this point, with 200+ entries, flip-flops are not recommended,
instead these would be placed in a RAM of some sort. Since RAMs
come in 1KB and 2KB quanta; we have 1 of 2K and 1 of 1K and we
have 256 said message containers, with 1 cycle access (after
you get to that corner of some chip).

> Keep in mind that a guest operating system may be writing
> the 64-bit address field in a virtual function assigned to
> it with guest virtual addresses, so the inbound path for
> MSI-X needs to traverse the IOMMU before hitting the
> interrupt controller logic.

An important concept that is hard to find in the PCIe specifications.

>                              The guest OS also specifies
> the data payload, which may be an IRQ number on an intel
> system, or a virtual IRQ number translated by the IOMMU
> or Interrupt controller into a physical IRQ number (allowing
> multiple guest OS to use the same IRQs they would use on real
> hardware).

Just note that GuestOS[k].IRQ[j] uses the same bit patterns
at the device, they get delivered to their own unique
GuestOS through its interrupt table.

But even when GuestOS[k] and GuestOS[j] use the same 96-bit MSI-X
pattern, each GuestOS can interpret its message the way it wants,
sort its drivers in the way it wants, and operate with blissful
ignorance of GuestOS[other].