Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail From: Bart Newsgroups: comp.lang.c Subject: Re: multi bytes character - how to make it defined behavior? Date: Wed, 14 Aug 2024 16:34:07 +0100 Organization: A noiseless patient Spider Lines: 89 Message-ID: References: <87sev8eydx.fsf@nosuchdomain.example.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Date: Wed, 14 Aug 2024 17:34:07 +0200 (CEST) Injection-Info: dont-email.me; posting-host="876cd8a29adc81823b5bb07945ef8107"; logging-data="545970"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/fEsqchUFv0ZM5qsILmjP3" User-Agent: Mozilla Thunderbird Cancel-Lock: sha1:3xnsYQO2yXVad/W2p5pfncPIeE4= In-Reply-To: Content-Language: en-GB Bytes: 4328 On 14/08/2024 14:31, Thiago Adams wrote: > On 14/08/2024 10:05, Bart wrote: >> On 14/08/2024 12:41, Thiago Adams wrote: >>> On 13/08/2024 21:33, Keith Thompson wrote: >>>> Bart  writes: >>>> [...] >>>>> What exactly do you mean by multi-byte characters? Is it a literal >>>>> such as 'ABCD'? >>>>> >>>>> I've no idea what C makes of that, >>>> It's a character constant of type int with an implementation-defined >>>> value.  Read the section on "Character constants" in the C standard >>>> (6.4.4.4 in C17). >>>> >>>> (With gcc, its value is 0x41424344, but other compilers can and do >>>> behave differently.) >>>> >>>> We discussed this at some length several years ago. >>>> >>>> [...] >>> >>> >>> "An integer character constant has type int. The value of an integer >>> character constant containing >>> a single character that maps to a single value in the literal >>> encoding (6.2.9) is the numerical value >>> of the representation of the mapped character in the literal encoding >>> interpreted as an integer. >>> The value of an integer character constant containing more than one >>> character (e.g. ’ab’), or >>> containing a character or escape sequence that does not map to a >>> single value in the literal encoding, >>> is implementation-defined. If an integer character constant contains >>> a single character or escape >>> sequence, its value is the one that results when an object with type >>> char whose value is that of the >>> single character or escape sequence is converted to type int." >>> >>> >>> I am suggesting the define this: >>> >>> "The value of an integer character constant containing more than one >>> character (e.g. ’ab’), or containing a character or escape sequence >>> that does not map to a single value in the literal encoding, is >>> implementation-defined." >>> >>> How? >>> >>> First, all source code should be utf8. >>> >>> Then I am suggesting we first decode the bytes. >>> >>> For instance, '×' is encoded with 195 and 151. We consume these 2 >>> bytes and the utf8 decoded value is 215. >> >> By that you mean the Unicode index. But you say elsewhere that >> everything in your source code is UTF8. > > > 215 is the unicode number of the character '×'. > >> Where then does the 215 appear? Do your char* strings use 215 for ×, >> or do they use 195 and 215? > > 215 is the result of decoding two utf8 encoded bytes. (195 and 151) > >> I think this is why C requires those prefixes like u8'...'. > >>> >>> Then this is the defined behavior >>> >>> static_assert('×' == 215) >> >> This is where you need to decide whether the integer value within >> '...', AT RUNTIME, represents the Unicode index or the UTF8 sequence. > > why runtime? It is compile time. This is why source code must be > universally encoded (utf8) In that case I don't understand what you are testing for here. Is it an error for '×' to be 215, or an error for it not to be? And what is the test for, to ensure encoding is UTF8 in this ... source file? ... compiler? Where would the 'decoded 215' come into it?