Deutsch English Français Italiano |
<09dac1eb164a4c5226036cbde84884da@www.novabbs.org> View for Bookmarking (what is this?) Look up another Usenet article |
Path: ...!weretis.net!feeder9.news.weretis.net!news.nk.ca!rocksolid2!i2pn2.org!.POSTED!not-for-mail From: mitchalsup@aol.com (MitchAlsup1) Newsgroups: comp.arch Subject: Re: PCIe MSI-X interrupts Date: Thu, 27 Jun 2024 01:47:49 +0000 Organization: Rocksolid Light Message-ID: <09dac1eb164a4c5226036cbde84884da@www.novabbs.org> References: <bb16865f7675526d4e2b87283e28c2c5@www.novabbs.org> <sKmdO.62321$G9_a.28048@fx13.iad> <bejeO.24905$Gurd.6236@fx34.iad> <8JkeO.30075$WaKd.3069@fx41.iad> <f9e2c222349934ce0d6d9cda5b55f8b7@www.novabbs.org> <FNzeO.141608$Cqra.114681@fx10.iad> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: i2pn2.org; logging-data="1305272"; mail-complaints-to="usenet@i2pn2.org"; posting-account="65wTazMNTleAJDh/pRqmKE7ADni/0wesT78+pyiDW8A"; User-Agent: Rocksolid Light X-Rslight-Posting-User: ac58ceb75ea22753186dae54d967fed894c3dce8 X-Spam-Checker-Version: SpamAssassin 4.0.0 X-Rslight-Site: $2y$10$AFjLVFUXj2CMoEMu4rWwuO0vQcMBRNfFngNkRvx0QgX71cMRi.VG2 Bytes: 4563 Lines: 80 Scott Lurndal wrote: > mitchalsup@aol.com (MitchAlsup1) writes: >>Scott, > Note that the ECAM must support 8, 16, 32-bit (optionally 64-bit) > single-copy atomic accesses to all configuration space registers. MMI/O is sequentially consistent while Config is Strongly ordered. Exactly what are you intending to mean from "single-copy atomic accesses" ?? Load Multiple (LM) instruction provides ATOMIC access to a series of sequentially ordered bytes in MMI/O (or Config, or DRAM). So, if desired, one could read 0x0-0x40 configuration header in a single LM. This is completely ATOMIC without SW doing anything other than supplying an address and a count. Memory to Memory Move (MM) is similar but reads from one place and writes to another in a single interconnect transaction--that is ATOMICally. {Basically as long as no page boundaries are crossed, each memory reference instruction is ATOMIC with respect to interested 3rd party observations.} Likewise, LDB, LDH, LDW, LDD and their ST counterparts are unit ATOMIC. But I don't know what you mean by "single-copy atomic accesses" ?? <snip> > That depends on how your interrupt controller is designed. If > you have a multi-socket/multi-chiplet configuration where all > chiplets are identical, and each has its own interrupt controller > (allowing single chiplet implementations), then you'll probably > want to use your CHIP bits in the address to route to the interrupt > controller on the closest chiplet just to reduce interrupt > latency. Yes, that was the concern. One might expect that the Guest OS would send an I/O request to a Guest OS device driver on the chip local to the PCIe tree the device is on, to minimize all the latency, not just interrupt delivery. Reading and writing of MMI/O space is <as they say> slow. > The interrupt controllers will likely need to cooperate > at the hardware level to maintain a single OS-visible "interrupt > space" where each controller handles a subset of the interrupt > number space. My model has multiple interrupt tables from the get go. For a start, I assume that each Guest OS has an interrupt table shared across however many number of virtual of physical cores the system manages. A HyperVisor has its own Interrupt Table, and the Secure Monitor has its own table. A core control register points at this table, and is used when negotiating for an interrupt, and used to detect Interrupt table priority escalation (when a priority bit is turned on). If SW wants a different Interrupt Table (or none) a simple write to the control register switches the table.* An interrupt table has interrupts raised when an MSI or MSI-X interrupt arrives at the Interrupt service provider port. I have configured the I/O MMU to translate DMA acceses through one set of MMU tables, and translate Interrupt access through <a conceptually> different MMU tables, and have access to the priority of the interrupt without taking MSI-X message bits. So, DMA can read or write directly through application MMU tables while the associated Interrupt goes to Guest OS at priority of SW's choice using the Interrupt table in charge when the I/O was setup. ----------------------------------------------------------- (*) So a virtual machine with 17 virtual cores and accepting interrupts on only 5 of them will have 5 with IT set and 12 with IT set invalid.