Deutsch   English   Français   Italiano  
<v31rj5$o20$1@dont-email.me>

View for Bookmarking (what is this?)
Look up another Usenet article

Path: ...!news.mixmin.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: David Brown <david.brown@hesbynett.no>
Newsgroups: comp.lang.c
Subject: Re: C23 thoughts and opinions
Date: Mon, 27 May 2024 13:42:28 +0200
Organization: A noiseless patient Spider
Lines: 256
Message-ID: <v31rj5$o20$1@dont-email.me>
References: <v2l828$18v7f$1@dont-email.me>
 <00297443-2fee-48d4-81a0-9ff6ae6481e4@gmail.com>
 <v2lji1$1bbcp$1@dont-email.me> <87msoh5uh6.fsf@nosuchdomain.example.com>
 <f08d2c9f-5c2e-495d-b0bd-3f71bd301432@gmail.com>
 <v2nbp4$1o9h6$1@dont-email.me> <v2ng4n$1p3o2$1@dont-email.me>
 <87y18047jk.fsf@nosuchdomain.example.com>
 <87msoe1xxo.fsf@nosuchdomain.example.com> <v2sh19$2rle2$2@dont-email.me>
 <87ikz11osy.fsf@nosuchdomain.example.com> <v2v59g$3cr0f$1@dont-email.me>
 <87plt8yxgn.fsf@nosuchdomain.example.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Mon, 27 May 2024 13:42:29 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="7989d793cb4f5bb8f14025e7e8038ebf";
	logging-data="24640"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1837rvvhx4DvPpa2a1QCxmBBmdrR0PAqLs="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Cancel-Lock: sha1:wJc7X44wR2tOI0zOfz9WjrR5lVI=
Content-Language: en-GB
In-Reply-To: <87plt8yxgn.fsf@nosuchdomain.example.com>
Bytes: 13825

On 27/05/2024 01:17, Keith Thompson wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> On 26/05/2024 00:58, Keith Thompson wrote:
>>> David Brown <david.brown@hesbynett.no> writes:
>>>> On 25/05/2024 03:29, Keith Thompson wrote:
>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>>>>>> David Brown <david.brown@hesbynett.no> writes:
>>>>>>> On 23/05/2024 14:11, bart wrote:
>>>>>> [...]


>>
>> The compiler will generate results /as if/ it had expanded the file to
>> a list of numbers and parsed them.  But it will not do that in
>> practice. (At least, not for more serious implementations - simple
>> solutions might do so to get support implemented quickly.)
> 
> I'll start by acknowledging that the prototype information apparently
> *does* optimize #embed when it can.  I was mistaken on that point.
> 
> #embed *must* expand to the standard-defined comma-delimited sequence in
> *some* cases.
> 
> Which means that the piece of the compiler that implements #embed has to
> recognize when it must generate that sequence, and when it can do
> something more efficient.

Yes, exactly.


>> I'd expect implementations to have extremely fast implementations for
>> initialising arrays of character types, and probably also for other
>> arrays of scaler types.  More complicated examples - such as
>> parameters in a macro or function call - would probably use a
>> fall-back of generating naïve lists of integer constants.
> 
> My problem is not just with how the compiler can figure out when it can
> optimize, but how programmers are supposed to understand whatever rules
> it uses.  Can I rely on the optimization being performed if I use a
> typedef for unsigned char, or if I use an enumeration type whose
> underlying type is unsigned char, or if I have initialization elements
> befor and after the #embed directive?

I don't know if that is something the programmer should need to 
consider, at least for most cases.  Generally as a programmer you don't 
consider the compilation speed when writing code.  You simply expect 
that compiler writers try to make their tools as fast as reasonably 
possible without sacrificing features.  Sometimes there can be 
particular use-cases where the programmer has to look at the compiler 
manuals and adapt the code or build procedures to suit.  I think that 
will be the case here too - compiler manuals should document what types 
of #embed usage they optimise.  But I think it is unlikely that people 
writing portable code will do anything other than initialising a const 
(or constexpr) array of unsigned char if they have big enough files for 
optimisation to be relevant.  Any compiler that does any #embed 
optimisation will handle this case.  And even simple #embed 
implementations will likely be better than any alternatives (such as 
using xxd).

> 
> Effective use of #embed requires too much "magic" for my taste --
> particularly having the preprocessor rely on information from later
> phases.  The semantics of #embed don't rely on that information, but
> efficient use for large files does.
> 

It is a violation of the neat layered (or pipeline) view of C 
compilation.  But you could argue that this has been broken for decades 
- you have _Pragma that is syntactically an operator but duplicates 
preprocessor work, you have compiler pragmas that duplicate command-line 
flags (and command-line flags that duplicate preprocessor defines), you 
have pre-compiled headers, you have LTO that passes data multiple times 
through different parts of the pipeline.

>>> If you have a binary file containing a sequence of int values, you
>>> can
>>> use #embed to initialize an unsigned char array that's aliased with or
>>> copied to the int array.
>>> The *embed element width* is typically going to be CHAR_BIT bits by
>>> default.  It can only be changed by an *implementation-defined* embed
>>> parameter.  It seems odd that there's no standard way to specify the
>>> element width.
>>> It seems even more odd that the embed element width is
>>> implementation defined and not set to CHAR_BIT by default.
>>
>> I agree.  But it may be left flexible for situations where the host
>> and target have different ideas about CHAR_BIT.  (Targets with
>> CHAR_BIT other than 8 are very rare, hosts with CHAR_BIT other than 8
>> are non-existent, but C remains flexible.)
> 
> I would think that you'd want the element width to match CHAR_BIT *on
> the target* (which is the only CHAR_BIT that's relevant or available).
> If you're cross-compiling, you'd probably want to embed a file that
> could have been used on the target system.

Yes, I think so.

> 
> And if I'm not doing that kind of exotic cross-compiling, I can't rely
> on the element width being CHAR_BIT *or* on any standard way to specify
> that I want it to be CHAR_BIT.
> 
> Requiring the default width to be CHAR_BIT would, I'm guessing, solve
> 99% of cases.  Allowing it to be specified by a parameter would solve
> the remaing 1%.  And I expect it *will* be CHAR_BIT in most or all
> implementations, and programmers will rely on that assumption.  I think
> the standard should guarantee that.
> 

I agree with you.  I'm just trying to think of why the standards might 
not make that guarantee.

>>> For a very large file, that could be a significant burden.  (I don't
>>> have any numbers on that.)
>>
>> I do :
>>
>> <https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3017.htm#design-efficiency-metrics>
>>
>> (That's from a proposal for #embed for C and C++.  Generating the
>> numbers and parsing them is akin to using xxd.)
>>
>> More useful links:
>>
>> <https://thephd.dev/embed-the-details#results>
>> <https://thephd.dev/implementing-embed-c-and-c++>
>>
>> (These are from someone who did a lot of the work for the proposals,
>> and prototype implementations, as far as I understand it.)
> 
> That second link does have a lot of good information.  I think I had
> seen it before, but I hadn't read it thoroughly.  It refers to prototype
> implementations for both gcc and clang.  I've built the prototype on my
> system, and godbolt.org has it, but the gcc prototype (for which the
> article provides good performance data) doesn't seem to be available
> anywhere.
> 

You are putting a lot more effort into this testing than I have.  For my 
work, I am generally dependent on "official" toolchain builds - provided 
by the manufacturers of the microcontrollers we use, or at least by the 
manufacturers of the cpu cores.  I like to keep track of what's coming - 
future versions of C or C++, future versions of compilers, etc.  But 
details such as implementation efficiency (rather than features) don't 
matter much to me until they are available as part of these pre-built 
toolchains.  (Sometimes it's fun to try things earlier, and I enjoy 
playing with newer compilers on godbolt.org, but I don't see testing the 
speed of #embed to be /so/ much fun that I'd bother building a compiler 
for it!)

But it's nice to see you've done some independent testing.  I have no 
particular reason to double "thephd.dev", but no particular reason to 
consider it authoritative either.

> My experiments with the clang prototype have been a bit confusing.  I
> assumed that `clang -E` would give me meaningful results, but it always
> produces the comma-delimited sequence of integer constants, and even
> that output is inconsistent.  It looks like "-E" synthesizes naive and
> not entirely correct output.  Feeding that output to clang produces
> warnings that I don't get without "-E".  Some of this might be the
> result of user error on my part.
> 
> I did some tests with 100MB file, both with #embed and with #include
> using the output of "xxd".  #embed *is* much faster.
> 
> According to <https://thephd.dev/implementing-embed-c-and-c++>, it
> internally generates __builtin_pp_embed, which takes as arguments the
> expected type (always unsigned char for now), the filename as a string
> literal, and the data encoded as a base64 string literal.  That's not
> going to be as fast as a hypothetical pure binary blob, but apparently
========== REMAINDER OF ARTICLE TRUNCATED ==========