Path: news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com>
Newsgroups: comp.lang.c
Subject: Re: Python recompile
Date: Fri, 14 Mar 2025 19:38:14 -0700
Organization: A noiseless patient Spider
Lines: 42
Message-ID: <vr2p6n$2hq49$4@dont-email.me>
References: <vq1qas$j22$1@gallifrey.nk.ca> <vqnfvp$1gsl4$2@dont-email.me>
 <20250310135828.116@kylheku.com> <vqnkdq$1gsl4$6@dont-email.me>
 <vqnmbn$1i1s0$6@dont-email.me> <vqnmig$1ie8j$2@dont-email.me>
 <vqns51$1jibk$5@dont-email.me> <vqo5hd$1lant$1@dont-email.me>
 <vqo9df$1ls6a$2@dont-email.me> <vqoe8o$1qblf$1@dont-email.me>
 <vqolqv$1rfnf$5@dont-email.me> <vqq8v2$262gq$1@dont-email.me>
 <vqqf3r$273mt$6@dont-email.me> <vqqjra$286h8$2@dont-email.me>
 <vqqlff$28fls$7@dont-email.me> <vqqtjv$2a3dv$1@dont-email.me>
 <vqqu21$2a18r$3@dont-email.me> <vqr25u$2eit2$1@dont-email.me>
 <vqras6$2fsqk$1@dont-email.me> <vqtq8v$33itc$1@dont-email.me>
 <vqvfl6$3rsid$2@dont-email.me> <vqvqmq$49pc$1@dont-email.me>
 <vr01sp$9jrt$2@dont-email.me> <vr212d$1uc77$1@dont-email.me>
 <vr26ih$23hho$1@dont-email.me> <vr299n$25gok$1@dont-email.me>
 <vr2ckr$2872d$2@dont-email.me> <vr2dts$29d9l$1@dont-email.me>
 <vr2kda$2ea00$4@dont-email.me> <vr2ooi$2hq49$1@dont-email.me>
 <vr2oun$2hq49$2@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 15 Mar 2025 03:38:15 +0100 (CET)
Injection-Info: dont-email.me; posting-host="f54594f8eb80134925da391bfb78d663";
	logging-data="2680969"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18Xb/tJRT2moRb1EetLc9XqJFap6BPagsM="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:wZUToiO7RqQkXmvTz/rz1MPodTQ=
In-Reply-To: <vr2oun$2hq49$2@dont-email.me>
Content-Language: en-US

On 3/14/2025 7:33 PM, Chris M. Thomasson wrote:
> On 3/14/2025 7:30 PM, Chris M. Thomasson wrote:
>> On 3/14/2025 6:16 PM, Lawrence D'Oliveiro wrote:
>>> On Fri, 14 Mar 2025 16:25:48 -0700, Chris M. Thomasson wrote:
>>>
>>>>> This kind of thing does not scale to having thousands of connections
>>>>> open at once.
>>>>
>>>> Yawn. Of course it does! 50,000 concurrent connections way back in 
>>>> early
>>>> 2000's.
>>>
>>> So, you have, what is it, 50,000 concurrent read and/or 50,000 
>>> concurrent
>>> write requests in flight at once?
>>>
>>> By the way, the default “hard” process open-file limit on this system 
>>> I’m
>>> using is half a million.
>>
>> It was about how to squeeze in 50,000 concurrent connections on some 
>> (now) older hardware. There were many tricks... One of them being 
>> zero- byte receives wrt IOCP. Keep in mind that IOCP can mess around 
>> with non- paged memory. So, you have to be very careful! You are 
>> reminding me of a so-called panic mode when the server would get to a 
>> point where shit might hit the fan, do I would dump timedout 
>> connections, dump as much cache as I could, but still try to maintain 
>> up time during times of really heavy load. It was an interesting time. 
>> 23-24 years ago.
> 
> I remember having a timeout thread that would see if a connection was in 
> a stale condition. Each per-socket connection would have a current state 
> and a sequence counter. Heck, in one of my very early tests I would 
> intentionally try to crash it. If it did crash due to non-paged memory, 
> malloc returning NULL, ect... It would be saving its state. Then I would 
> see how the system went down. Try again with a threshold and see if I 
> could do it again. Just experimenting and testing load.

LOL! I remember malloc returning NULL as a trigger for panic mode. It 
would dump as much state as it could, then try the malloc again. On some 
tests I would allocate a massive block of memory up front and use a 
simple region allocator for it.