Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Robert Finch <robfi680@gmail.com>
Newsgroups: comp.arch
Subject: Re: MSI interrupts
Date: Tue, 18 Mar 2025 17:59:51 -0400
Organization: A noiseless patient Spider
Lines: 102
Message-ID: <vrcqcr$3ddah$1@dont-email.me>
References: <vqto79$335c6$1@dont-email.me>
 <53b8227eba214e0340cad309241af7b5@www.novabbs.org>
 <3pXAP.584096$FVcd.26370@fx10.iad>
 <795b541375e3e0f53e2c76a55ffe3f20@www.novabbs.org>
 <vNZAP.37553$D_V4.18229@fx39.iad>
 <aceeec2839b8824d52f0cbe709af51e1@www.novabbs.org>
 <eM_AP.81303$8rz3.7843@fx37.iad> <vr2nj9$2goqe$1@dont-email.me>
 <f2cb846242dbfcef1efa59b92763a965@www.novabbs.org>
 <vr4ovm$9fl5$1@dont-email.me>
 <1681197d3c1af131d6b8cae884f7c9ca@www.novabbs.org>
 <vr7g76$2jnqm$1@dont-email.me> <8BVBP.816276$eNx6.247046@fx14.iad>
 <20250317161132.00004dd9@yahoo.com> <1WZBP.558392$SZca.243157@fx13.iad>
 <vra3bv$teuf$1@dont-email.me> <ohjCP.37185$SVG3.13173@fx42.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 18 Mar 2025 22:59:56 +0100 (CET)
Injection-Info: dont-email.me; posting-host="5cd78d03bd7d745af32885f38519e6cb";
	logging-data="3585361"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1+C1sJ5k8cNmathx6jqzn/MoqL2Zt3pmT8="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:Tprs9xyDOl2McizE4PcZkpo0cKo=
In-Reply-To: <ohjCP.37185$SVG3.13173@fx42.iad>
Content-Language: en-US
Bytes: 6850

On 2025-03-18 2:51 p.m., EricP wrote:
> Robert Finch wrote:
>> On 2025-03-17 2:33 p.m., EricP wrote:
>>>
>>> Another problem is what does the core do with the in flight 
>>> instructions.
>>>
>>>
>>> Method 2 pipelines the switch by injecting the interrupt request at 
>>> Fetch.
>>> Decode converts the request to a special uOp that travels down the IQ
>>> to Retire and allows all the older work to complete.
>>> This is more complex as it requires a two phase hand-off from the
>>> Interrupt Control Unit (ICU) to the core as a branch mispredict in the
>>> in flight instructions might cause a tentative interrupt acceptance 
>>> to later
>>> be withdrawn.
>>
>> Oh my, I forgot.
>> Method 2 will be used as it is desired to able to insert either an 
>> instruction at the fetch stage or a vector address.
>>>
>>> The ICU believes the core is in a state to accept a higher priority
>>> interrupt. It sends a request to core, which checks its current state 
>>> and
>>> sends back an immediate INT_ACK if _might_ accept and stalls Fetch, 
>>> or a NAK.
>>> When the special uOp reaches Retire, it sends a signal to Fetch which
>>> then sends an INT_ACCEPT signal to ICU to complete the handoff.
>>> If a branch mispredict occurs that causes interrupts to be disabled,
>>> then Fetch sends an INT_REJECT to ICU, and unstalls its fetching.
>>> (Yes that is not optimal - make it work first, make it work well 
>>> second.)
>>>
>>> This also raises a question about what the ICU is doing during this
>>> long latency handoff. One wouldn't want ICU to sit idle so it might
>>> have to manage the handoff of multiple interrupts to multiple cores
>>> at the same time, each as its own little state machine.
>>>
>>> One should see that this decision on how the core handles the
>>> handoff has a large impact on the design complexity of the ICU.
>>>
>>> Personally, I would use method 1 first to get something working
>>> then if this is OoO think about getting fancy with method 2.
>>>
>>>
>> Would it not be rare for this to occur? If so, I think the pipeline 
>> could just be flushed from the int disabling instruction on. Then the 
>> interrupted address recorded as the int disabling instruction. <- this 
>> requires identification of an int disabling instruction though. It 
>> might just be a store to a specific I/O address, therefore a special 
>> store opcode to make it easy to detect?
> 
> It likely is rare but the possibility that the existing instructions,
> a branch mispredict or an exception throw, could retroactively change
> the state of the interrupt enable flag to disabled creates this
> Window of Uncertainty (WoU) that is the length of the pipeline.
> 
> Unless one can find a way to make WoU go away the ICU needs to deal
> with the possibility that there could be hundreds of clocks between
> when ICU asks a core if it can accept an interrupt and when the core does.
> 
> I had a few ideas walking in the park yesterday. I had been assuming that
> when an exception is delivered that it would also automatically disable
> interrupts. That may not be a good idea as it means that any instruction
> that can throw an exception could cause this retroactive disabling.
> 
> If I remove that exception side effect then the only things that can
> manipulate the core's master Interrupt Enable flag are the enable INTEN
> and disable INTDI instructions, and the actual delivery of an interrupt.
> And those three things can be tracked with a counter in Decode as they
> move down the pipeline to Retire. If the counter >0 then there is a
> flag change in flight and we don't accept a new interrupt. When count == 0
> and Interrupt Enable flag == 1 then core accepts interrupts, which are
> injected at Decode and cause a light weight purge of the Fetch buffer.
> 
Seems like it could work. There are difficulties using counters with the 
Q+ ISA as multiple interrupt enable bits could be manipulated at the 
same time. One interrupt may be enabled and another one disabled by the 
same instruction.

I am going for a simple approach. Since the CSR instruction is used to 
manipulate the interrupt enable bits in the status register, if there is 
any CSR instruction in the pipeline, then interrupts will be masked off 
until the CSR clears. One issue with this type of approach is evil 
software could issue a lot of CSRs preventing interrupt servicing from 
happening. I Could fine tune the decode to sort out CSRs just affecting 
the interrupt flags. It always seems to be more logic.

> Another pipeline issue is Privileged Control Register (PCR) reads and 
> writes.
> PCR's don't behave like general registers in that they are not renamed,
> and in general operations cannot be replayed or performed out of order.
> They behave more like atomic stores that must be performed at Retire
> and must not be bypassed by any memory and other PCR ops. (This is why
> many of the x86/x64 MSR ops are serialized with a pipeline drain.)
> 
> This affects what it can do with PCR inside the WoU if it causes a state
> change that cannot be undone, for example, reading the interrupt data FIFO.
> 
>