Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: BGB <cr88192@gmail.com>
Newsgroups: comp.lang.c
Subject: =?UTF-8?Q?Re=3A_technology_discussion_=E2=86=92_does_the_world_need?=
 =?UTF-8?B?IGEgIm5ldyIgQyA/?=
Date: Tue, 9 Jul 2024 13:23:15 -0500
Organization: A noiseless patient Spider
Lines: 148
Message-ID: <v6jv6k$1fhoh$1@dont-email.me>
References: <v66eci$2qeee$1@dont-email.me> <v67gt1$2vq6a$2@dont-email.me>
 <v687h2$36i6p$1@dont-email.me> <871q48w98e.fsf@nosuchdomain.example.com>
 <v68dsm$37sg2$1@dont-email.me> <87wmlzvfqp.fsf@nosuchdomain.example.com>
 <v6ard1$3ngh6$4@dont-email.me> <v6b0jv$3nnt6$1@dont-email.me>
 <v6c298$3tko2$1@dont-email.me> <v6c688$3uf4o$1@dont-email.me>
 <v6jhvb$1drd6$2@dont-email.me> <20240709185406.0000734f@yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 09 Jul 2024 20:23:16 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="05b7ca2b320d2b6fc0402746ff5c99a9";
	logging-data="1558289"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1+3Z2rbQRTRPklu7nWlh/nnh+iKaqEIwjo="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:KNOHKoxS35XRaJTsZl9Zo4vFo8s=
Content-Language: en-US
In-Reply-To: <20240709185406.0000734f@yahoo.com>
Bytes: 6836

On 7/9/2024 10:54 AM, Michael S wrote:
> On Tue, 9 Jul 2024 16:37:31 +0200
> David Brown <david.brown@hesbynett.no> wrote:
> 
>> On 06/07/2024 21:33, BGB wrote:
>>>
>>> In my compiler (BGBCC), such an internal pointer exists for arrays
>>> and structures in the local stack frame.
>>>
>>> No separate pointer exists inside of things like structs, where, as
>>> can be noted, the array exists at a fixed size and location.
>>>
>>>
>>> So, eg:
>>>     void Foo()
>>>     {
>>>        int a[100];
>>>        ...
>>>     }
>>>
>>> There is both the space for 100 integers reserved in the stack
>>> frame, and a variable 'a' which exists as an implicit pointer to
>>> that location.
>>>
>>>
>>> But, say:
>>>     void Foo()
>>>     {
>>>        int a[8192];
>>>        ...
>>>     }
>>>
>>> There is no space reserved on the stack, and the array is instead
>>> allocated dynamically (from the heap). In this case, the "a"
>>> variable exists as a pointer to that location in memory.
>>>
>>> Similar treatment also applies to structs.
>>>    
>>
>>
>> The C standard does not require a stack or say how local data is
>> implemented, it just gives rules for the scope and lifetime of
>> locals. However, I would be surprised and shocked to find a compiler
>> I was using allocate local data on the heap in some manner.  If I
>> have an array as local data, it is with the expectation that it is
>> allocated and freed trivially (an add or subtract to the stack
>> pointer, typically combined with any other stack frame).  If I want
>> something on the heap, I will use malloc and put it on the heap.
>>
>> Such an implementation as yours is not, I think, against the C
>> standards
>> - but IMHO it is very much against C philosophy.
>>
> 
> I wouldn't mind if my ABI/compiler allocates all big local objects
> together with all local VLA either on separate secondary stack or even
> on heap. Such strategy will improve locality of reference for primary
> stack. So, despite higher management overhead, it could sometimes be
> advantageous even for performance.
> The main advantage however is not performance, but reducing the
> severity of damage caused by buffer overrun bugs.
> 
> Although if "security" is a primary concern then one would want
> stricter policy than the one outlined above. I.e. not only big objects,
> but small object as well should be allocated away from primary stack as
> long as their address participate in pointer arithmetic that can't be
> proven safe by static analysis.
> 

It is mostly for big objects to not lead to stack overflows...

On a 128K stack, theoretically the maximum number of 16K objects on the 
stack is 8 (assuming no space for anything else).

You don't want to require a big stack if:
It is possible for the system to run without an MMU, thus bigger stacks 
waste memory;
Most of the programs do not benefit from having a larger stack.


I did tweak the rules slightly a little more recently:
   If an object is 16K or larger, it goes on the heap;
   If the sum of objects is larger than 16K:
     Calculate n = total / 12K
     New limit is 16K/n.

So, say:
   int arr1[3072];
   int arr2[3072];
Previously, both could go on the stack (using 24K), but now they will 
trigger the limit to be reduced to 8K, putting both on the heap.


Though, as noted, these objects go into a linked list and will be 
automatically freed when the function returns.

I recently did modify the mechanism (moving the responsibility from the 
frontend to the backend frame-layout handling), which should hopefully 
reduce some of the bugs I was seeing with it.


Partly, what triggered some of this originally, was GLQuake had a buffer 
like:
   unsigned resampbuf[256*256];

Which, would have worked with a 1MB stack, but with a 128K stack led to 
an immediate overflow.

This mechanism still allows such code to exist...

For context, the array was used when uploading textures to OpenGL, as 
Quake used many non-power-of-2 textures, but OpenGL only allows 
power-of-2 texture sizes; with an accepted maximum of 256*256 as this 
was the limit on early graphics cards; though, newer ones can typically 
handle 4096*4096 or similar; currently TKRA-GL assumes a limit of 1024*1024.



The general assumption though, is that these sort of large stack objects 
are the exception rather than the rule, so a little extra overhead is 
tolerable (but, implicitly, things like interrupt handlers, etc, are not 
allowed to use large stack arrays).

Well, more so that in my case the interrupt handler currently has a 64K 
stack. Originally, it was ~ 6K (in a special SRAM area), but it was 
difficult to avoid overflows (with things like virtual memory code), so 
I ended up moving the interrupt stack to RAM.



Based on some looking, apparently early versions of Mac OS/X may have 
done something similar, albeit:
   With an object-size limit of 32K;
   With a stack size of 512K.

This isn't too far off in terms of proportions.

Though, it looks like more modern versions went to using bigger stacks 
(no information if they still do the array folding).

Can note that:
   Windows defaults to a 1MB stack size;
   Linux defaults to an 8MB stack size.
   Appears modern OSX is 4MB.

Granted, none of these systems is NOMMU.