Path: news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Robert Finch <robfi680@gmail.com>
Newsgroups: comp.arch
Subject: Re: MSI interrupts
Date: Thu, 13 Mar 2025 13:49:35 -0400
Organization: A noiseless patient Spider
Lines: 78
Message-ID: <vqv5rj$3kaa7$1@dont-email.me>
References: <vqto79$335c6$1@dont-email.me>
 <3d5200797dd507ae051195e0b2d8ff56@www.novabbs.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Thu, 13 Mar 2025 18:49:40 +0100 (CET)
Injection-Info: dont-email.me; posting-host="134c048fe25441bee20a3d0d36ac0fa1";
	logging-data="3811655"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/UOoq5/UJfJQgsABsgGHxexCLC1Am2OD0="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:OZFmkWVg+8VO9nTdVDbut+ebPf8=
Content-Language: en-US
In-Reply-To: <3d5200797dd507ae051195e0b2d8ff56@www.novabbs.org>

On 2025-03-13 12:34 p.m., MitchAlsup1 wrote:
> On Thu, 13 Mar 2025 4:50:47 +0000, Robert Finch wrote:
> 
>> Planning on having the Q+ CPU respond directly to MSI interrupts. My
>> thought is to integrate an interrupt controller directly into the CPU
>> which will have several queues to prioritize interrupts. The controller
>> will take care of detecting the highest priority interrupt to process,
>> performing priority inversion occasionally, and detect stuck interrupts.
> 
> My 66000 went all the way to MSI-X interrupts as its base.
> 
> There are mapping methods to map INTx and INTk into MSI-X messages.
> MSI->MSI-X mapping is "just zero extend the message".

MSI interrupts are the way to go when there are transistors available. 
Just having discovered them not to long ago, I still think they are great.
I have the luxury of not needing to support legacy interrupt systems 
(yet), all the peripheral cores needing interrupts will have to support 
MSI responses.
> 
> The Interrupt port lives in the memory controller and takes an
> aperture of addresses and maps them to various interrupt tables.
> 
I have read that MSI devices target particular I/O addresses to send the 
messages to. Not sure what having a dedicated I/O addresses buys. In my 
architecture, rather than target an I/O address a specific processing 
core number is specified.

> There is a broadcast mechanism that notifies cores of new higher
> priority level interrupts pending, and a protocol for getting
> an interrupt and notifying the IC that it will be handled by a
> core.
> 
> Along with the 32-bit message, the architecture provides for
> 6-bits of priority, 2-bits of SW-stack indexing, and an 8-bit
> ISR handler index number.

My architecture goes with a 3-bit priority field, and an 8-bit field to 
help resolve the cause of the interrupt. The bus/device/function is also 
part of the interrupt response. A 6-bit target core number field is 
included in the response too. A target core number of all ones is 
treated as a broadcast to all cores. The entire interrupt response 
message is available to the CPU so software can decide how to handle the 
interrupt. There is some potential for data to be passed back. The 
contents of the data field passed back on the bus is unspecified. The 
other info is passed back in the address field of the bus.
> 
> Interrupt tables remain active even when the virtual machine
> it inactive. When next scheduled, priority ISRs will be serviced
> prior to low priority thread instructions.
> 
> {And some things I can't talk about}
Shucks.
> 
>> Setting up an MSI interrupt protocol for Q+.
>>
>> The controller will sit on the CPU's main response collect bus.
>> Interrupt messages contain the interrupter's bus/device/function + a
>> cause code. They may also contain a target core number.
>>
>> Planning on distributing the interrupt control to peripheral devices
>> instead of having a system level interrupt controller.

I sketched up an interrupt controller in SystemVerilog and it was only 
about 700 LUTs although it does use seven BRAMs for priority queues.
The priority queues can hold hundreds of interrupt messages which is 
likely far more than would be needed. So if a queue overflows it would 
indicate an issue with the interrupt system. The controller also keeps a 
message history with timestamps. If the same message is present at the 
start and end of the history along with a short difference in the 
timestamp, it is considered a stuck interrupt.

The controller inverts interrupt priorities for one cycle every 64 clock 
cycles. The priority inversion is with respect to which queue is read 
from. The idea is to try and prevent a low-priority task that is waiting 
on an interrupt from being permanently disabled.