Path: ...!weretis.net!feeder9.news.weretis.net!news.nk.ca!rocksolid2!i2pn2.org!.POSTED!not-for-mail
From: mitchalsup@aol.com (MitchAlsup1)
Newsgroups: comp.arch
Subject: Re: Stacks, was Segments
Date: Sat, 15 Feb 2025 23:28:28 +0000
Organization: Rocksolid Light
Message-ID: <fcaa9301fe9800a1aab0c4fddd0a7323@www.novabbs.org>
References: <vdlgl9$3kq50$2@dont-email.me> <H5upP.2$72m1.1@fx11.iad> <4a1eb52bbccf3c20554ac8016bb7f97e@www.novabbs.org> <0atqP.430$EyH6.259@fx45.iad> <fb334c4df427a76f8c12c8f2312a9022@www.novabbs.org> <fOIqP.674411$iNI.193453@fx14.iad> <fc735aa8a75233ef89a789a8ab8d92bc@www.novabbs.org> <oJOqP.674414$iNI.243253@fx14.iad> <968818239db4609805e5375d6aa814bd@www.novabbs.org> <6673961f57b047ec5ade59bf8d03c016@www.novabbs.org> <k9NrP.11363$z8ke.7319@fx15.iad> <73648ac951e13922bb386a8805e88d28@www.novabbs.org> <Ar2sP.919$wlqe.720@fx06.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
	logging-data="228337"; mail-complaints-to="usenet@i2pn2.org";
	posting-account="o5SwNDfMfYu6Mv4wwLiW6e/jbA93UAdzFodw5PEa6eU";
User-Agent: Rocksolid Light
X-Rslight-Site: $2y$10$org1QZToFbuc1JbehyNQUuIfTiD8i2SuqI/diRC8kaUsdnaQTtw7a
X-Rslight-Posting-User: cb29269328a20fe5719ed6a1c397e21f651bda71
X-Spam-Checker-Version: SpamAssassin 4.0.0
Bytes: 4630
Lines: 71

On Sat, 15 Feb 2025 15:31:44 +0000, Scott Lurndal wrote:

> mitchalsup@aol.com (MitchAlsup1) writes:
-----------
>>> PCI supports two forms of configuration space addresses in TLPs:
>>>    Type 0 contains only a function number and the register address.
>>>    Type 1 contains the bus number, function number and register address.
>>
>>PCIe segments go where ? Are they "picked off" prior to being routed
>>down the tree ??
>
> Logically, they can be considered a prefix to the RID for routing
> purposes (inbound to the IOMMU, for example, the PCIE controller
> will prepend its segment number to the RID and use that as an
> ID to the IOMMU).   ARM calls it a streamid.
>
> For PCI configuration transactions initiated by the CPU,
> "PCIe segments go where?" is an interesting question.

Indeed. Snipping Intel brain damage
-----------------------
> PCI Express has been designed as a point-to-point protocol
> using serial connections rather than the wide PCI local
> bus, which changes the topology of the system.   With PCIe
> the device number in the RID -must be zero- (unless the
> BUS is an ARI bus,

What is an ARI bus, and do non x86 systems have them ??

>                    in which case bits <7:0> of the RID
> are a function number provided by the PCIe device (up
> to 256 functions per each - more with SRIOV as it can
> consume additional space in the bus <15:8> field of the
> RID to support up to 65535 virtual functions on a single
> device).  A non-SRIOV and non-ARI device can only provide
> from one to eight functions.

snipping

> The specification allows the implementation to provide a single ECAM
> segment per PCIe controller, or and implementation may provide a single
> ECAM region and used bits <63:28> as a segment number.

Wikipedia states ECAM contains 42 bits::
16-bit segment, 8-bit bus, 5-bit device, 3-bit function, 4-bit
xReg, and 6-bit reg. Is this wrong, misleading, of out of date ?

>                                                       This is how
> most non-intel systems handle this today; an processors that supports
> six PCIe controllers would have perhaps 7 segments (one or more for the
> root bus 0 for the on-chip devices such as memory controllers, and
> one for each PCIe controller.
>
> Software simply constructs the target ECAM address and issues normal
> loads and stores to access it - no need to use the clumsy, slow and
> non-standard PCI peek-and-poke configuration space accesses.

We still have to translate 'target ECAM' into a bit pattern matching
that device's BAR. And thinks to your help I have a means to do so
that has overhead only when Booting a new Guest OS.

> For outbound memory space (non-config space) transactions from the CPU
> to
> the device, the RID is not used - once the CPU I/O fabric has routed
> the request by PA to the proper PCIe controller (address based routing
> through a heirarchy of host bridges), it is simply sent to the device
> which CAMs the address against the programmed bars and reacts
> appropriately
> (e.g. by responding with a UR (Unsupported Request) on a non-posted
> request or dropping a posted request if it doesn't match a BAR).

Agreed.