Path: news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Robert Finch <robfi680@gmail.com>
Newsgroups: comp.arch
Subject: Re: MSI interrupts
Date: Thu, 13 Mar 2025 15:59:02 -0400
Organization: A noiseless patient Spider
Lines: 102
Message-ID: <vqvde9$3qe81$1@dont-email.me>
References: <vqto79$335c6$1@dont-email.me>
 <3d5200797dd507ae051195e0b2d8ff56@www.novabbs.org>
 <vqv5rj$3kaa7$1@dont-email.me> <pGFAP.729660$eNx6.592144@fx14.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Thu, 13 Mar 2025 20:59:06 +0100 (CET)
Injection-Info: dont-email.me; posting-host="134c048fe25441bee20a3d0d36ac0fa1";
	logging-data="4012289"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1+SbIbof9fz/C4yuCE8RJE0qhiZ7O9winc="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:IO6CBSmrODjVPZHXR6hzZ6gismw=
Content-Language: en-US
In-Reply-To: <pGFAP.729660$eNx6.592144@fx14.iad>

On 2025-03-13 2:42 p.m., Scott Lurndal wrote:
> Robert Finch <robfi680@gmail.com> writes:
>> On 2025-03-13 12:34 p.m., MitchAlsup1 wrote:
>>> On Thu, 13 Mar 2025 4:50:47 +0000, Robert Finch wrote:
>>>
>>>> Planning on having the Q+ CPU respond directly to MSI interrupts. My
>>>> thought is to integrate an interrupt controller directly into the CPU
>>>> which will have several queues to prioritize interrupts. The controller
>>>> will take care of detecting the highest priority interrupt to process,
>>>> performing priority inversion occasionally, and detect stuck interrupts.
>>>
>>> My 66000 went all the way to MSI-X interrupts as its base.
>>>
>>> There are mapping methods to map INTx and INTk into MSI-X messages.
>>> MSI->MSI-X mapping is "just zero extend the message".
>>
>> MSI interrupts are the way to go when there are transistors available.
>> Just having discovered them not to long ago, I still think they are great.
>> I have the luxury of not needing to support legacy interrupt systems
>> (yet), all the peripheral cores needing interrupts will have to support
>> MSI responses.
> 
> MSI has been deprecated for years.  MSI-X is preferred and used by
> most modern devices (and supported by major operating software).
> 
>>>
>>> The Interrupt port lives in the memory controller and takes an
>>> aperture of addresses and maps them to various interrupt tables.
>>>
>> I have read that MSI devices target particular I/O addresses to send the
>> messages to. Not sure what having a dedicated I/O addresses buys. In my
>> architecture, rather than target an I/O address a specific processing
>> core number is specified.
> 
> There are many reasons that MSI-X supports vector specific target addresses.
>   1) It is possible to target DRAM (so software can poll for the interrupt)

This can be done using a dummy core in the system that echos interrupts 
messages back to the DRAM. A current issue with the way interrupts in my 
system are working, is they always spit out the interrupt on the 
response bus, which means they cannot act like the master.

To drive messages onto the bus as requests instead of responses every 
peripheral device would need to be able to be a bus master. It would add 
a lot of logic to the system. Extra cross-bar ports etc.


>   2) It is possible to support level-sensitive semantics by using two vectors;
>      one to assert the interrupt and the second to deassert it.  Each needs
>      to target a different interrupt controller register (address).

I think this can be handled with the cause code being returned as part 
of the response.

>   3) A system may have multiple interrupt controllers for different
>      security states (e.g. monitor/secure vs. hypervisor vs. direct virtual
>      to a guest).

Is this just to get more target vectors for each interrupt? Could there 
be one interrupt controller with multiple target vectors depending on 
the security state?

Q+ CPU always switches to machine mode to process interrupts. It can 
then redirect to other operating modes. A security state code could be 
added to the response so that different vector tables may be used.

> 
> 
>>
>> My architecture goes with a 3-bit priority field, and an 8-bit field to
>> help resolve the cause of the interrupt. The bus/device/function is also
>> part of the interrupt response.
> 
> That's what ARM64 GIC does, each interrupt is accompanied by a 20+ bit
> 'device id', which is composed of the segment, bus, dev/func values
> for the sending device.   Since each PCIe controller can consume the
> entire bus space potentially, the segment widens the device id space
> to include a large number of devices.   The 'device id' is used to
> select the in-memory table used to map the interrupt to the target
> guest and the target guest vector number.

Seems like going out to memory tables to fetch the targets of an 
interrupt would be costly, perhaps slow. In the SoC there are already a 
half dozen devices fighting for access to the DRAM.

> 
> I think you really want an interrupt controller as an intermediary
> between the device and the CPU interrupt signal, allows a great deal
> of implementation flexibility (e.g. changing the target CPU in the

Absolutely. Be crazy not to. There is an irq controller component 
separate from the CPU. (I decided to move the controller outside of the 
CPU).

> interrupt controller tables rather than requiring the device driver
> to change the msix-vector address field).
> 
Updating I/O is faster than updating DRAM tables.

I need to study more on the use of interrupt tables.