Path: ...!weretis.net!feeder9.news.weretis.net!panix!.POSTED.spitfire.i.gajendra.net!not-for-mail
From: cross@spitfire.i.gajendra.net (Dan Cross)
Newsgroups: comp.arch
Subject: Re: DMA is obsolete
Date: Sat, 3 May 2025 13:33:47 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <vv55vr$6hg$1@reader1.panix.com>
References: <vuj131$fnu$1@gal.iecc.com> <2025May2.073450@mips.complang.tuwien.ac.at> <vv2mqb$hem$1@reader1.panix.com> <2025May3.081100@mips.complang.tuwien.ac.at>
Injection-Date: Sat, 3 May 2025 13:33:47 -0000 (UTC)
Injection-Info: reader1.panix.com; posting-host="spitfire.i.gajendra.net:166.84.136.80";
	logging-data="6704"; mail-complaints-to="abuse@panix.com"
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: cross@spitfire.i.gajendra.net (Dan Cross)
Bytes: 23329
Lines: 486

In article <2025May3.081100@mips.complang.tuwien.ac.at>,
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>cross@spitfire.i.gajendra.net (Dan Cross) writes:
>>In article <2025May2.073450@mips.complang.tuwien.ac.at>,
>>Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>>>I think it's the same thing as Greenspun's tenth rule: First you find
>>>that a classical DMA engine is too limiting, then you find that an A53
>>>is too limiting, and eventually you find that it would be practical to
>>>run the ISA of the main cores.  In particular, it allows you to use
>>>the toolchain of the main cores for developing them,
>>
>>These are issues solveable with the software architecture and
>>build system for the host OS.
>
>Certainly, one can work around many bad decisions, and in reality one
>has to work around some bad decisions, but the issue here is not
>whether "the issues are solvable", but which decision leads to better
>or worse consequences.

I don't know that either would be "better" or "worse" under any
objective criteria.  They would simply be different.

>>The important characteristic is
>>that the software coupling makes architectural sense, and that
>>simply does not require using the same ISA across IPs.
>
>IP?  Internet Protocol?

When we discuss hardware designs at this level, reusable
components that go into the system are often referred to as "IP
cores" or just "IPs".  For example, a UART might be an IP.

Think of them as building blocks that go into, say, a SoC.

>Software Coupling sounds to me like a concept
>from Constantine out of my Software engineering class.

I have no idea who or what that is, but it seems unrelated.

>I guess you
>did not mean either, but it's unclear what you mean.

It's a very common term in this context.
https://en.wikipedia.org/wiki/Semiconductor_intellectual_property_core

>In any case, I have made arguments why it would make sense to use the
>same ISA as for the OS for programming the cores that replace DMA
>engines.  I will discuss your counterarguments below, but the most
>important one to me seems to be that these cores would cost more than
>with a different ISA.  There is something to that, but when the
>application ISA is cheap to implement (e.g., RV64GC), that cost is
>small; it may be more an argument for also selecting the
>cheap-to-implement ISA for the OS/application cores.

Ok.

>>Indeed, consider AMD's Zen CPUs; the PSP/ASP/whatever it's
>>called these days is an ARM core while the big CPUs are x86.
>>I'm pretty sure there's an Xtensa DSP in there to do DRAM and
>>timing and PCIe link training.
>
>The PSPs are not programmable by the OS or application programmers, so
>using the same ISA would not benefit the OS or application
>programmers.

Its firmware ships in BIOS images.  You can, in fact, interact
with it from the OS.  The only thing that keeps it from being
programmable by the OS is signing keys.

>By contrast, the idea for the DMA replacement engines is
>that they are programmable by the OS and maybe the application
>programmers, and that changes whether the same ISA is beneficial.
>
>What is "ASP/whatever"?

The PSP, or "AMD Platform Security Processor", has many names.
AMD says that "PSP" is the "legacy name", and that the new name
is ASP, for "AMD Secure Processor", and that it provides
"runtime security services"; for example, the PSP implements a
TPM in firmware, and exposes a random number generator that x86
can access via the `RDRAND` instruction.

>>Similarly with the ME on Intel.
>
>Last I read about it, ME uses a core developed by Intel with IA-32 or
>AMD64; but in any case, the ME is not programmable by OS or
>application programmers, either.

I was under the impression that it started out as an ARM core,
but I may be mistaken.

In any case, where do you think its firmware comes from?

>>A BMC might be running on whatever.
>
>Again, a BMC is not programmable by OS or application programmers.

The people working on OpenBMC disagree.

>>We increasingly see ARM
>>based SBCs that have small RISC-V microcontroller-class cores
>>embedded in the SoC for exactly this sort of thing.
>
>That's interesting; it points to RISC-V being cheaper to implement
>than ARM.  As for "that sort of thing", they are all not programmable
>by OS or application programmers, so see above.

No, the entire point is to provide an off-load for things that
are real-time.  They are absolutely meant to be "programmable by
OS or application programmers", which is exactly the sort of
scenario that Mitch's proposed cores would be used for.

Is a GPU programmable?  Yes.  Does it use the same ISA as the
general purpose compute core?  No.

>>Our hardware RoT
>
>?

Root of Trust.

>>The problem is when such service cores are hidden (as they are
>>in the case of the PSP, SMU, MPIO, and similar components, to
>>use AMD as the example) and treated like black boxes by
>>software.  It's really cool that I can configure the IO crossbar
>>in useful way tailored to specific configurations, but it's much
>>less cool that I have to do what amounts to an RPC over the SMN
>>to some totally undocumented entity somewhere in the SoC to do
>>it.  Bluntly, as an OS person, I do not want random bits of code
>>running anywhere on my machine that I am not at least aware of
>>(yes, this includes firmware blobs on devices).
>
>Well, one goes with the other.  If you design the hardware for being
>programmed by the OS programmers, you use the same ISA for all the
>cores that the OS programmers program,

That's a categorical statement that is not well supported.  That
may be what is _usually_ done.  It is not what _has_ to be done,
or even what _should_ be done.

You may feel that ths is the way things should be done, but the
arguments you've presented so far are not persuasive.

>whereas if you design the
>hardware as programmed by "firmware" programmers, you use a
>cheap-to-implement ISA and design the whole thing such that it is
>opaque to OS programmers and only offers some certain capabilities to
>OS programmers.

There is little fundamental difference between "firmware" and
the "OS".  I would further argue that this model of walling off
bits of system programmed with "firmware" from the OS a dated
way of thinking about systems that is actively harmful.  See
Roscoe's OSDI'21 keynote, here:

https://www.usenix.org/conference/osdi21/presentation/fri-keynote

Insisting that we use the congealed model we currently use
because that's how it is done is circular reasoning.

>And that's not just limited to ISAs.  A very successful example is the
>way that flash memory is usually exposed to OSs: as a block device
>like a plain old hard disk, and all the idiosyncracies of flash are
>hidden in the device behind a flash translation layer that is
>implemented by a microcontroller on the device.

You're conflating a hardware interface with firmware.

>What's "SMN"?

The "System Management Network."  This is the thing that AMD
uses inside the SoC to talk between the different components
that make up the system (that is, between the different IPs in
the SoC).  SMN is really a network of AXI buses, but it's how
one can, say, read and write registers on various components.

If you look at, for example,
https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/55803-ppr-family-17h-model-31h-b0-processors.pdf
And you look at the enry for the SMU registers, you'll see
that they have an "aliasSMN" entry in the instance table; those
can be decoded to a 32-bit number.  That is the SMN address of
that register.  For example, `SMU::THM::THM_TCON_CUR_TMP` is the
thermal register maintained by the SMU that encodes the current
temperature (in normalized units that are scaled from e.g.
========== REMAINDER OF ARTICLE TRUNCATED ==========