Path: ...!weretis.net!feeder8.news.weretis.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: David Brown <david.brown@hesbynett.no>
Newsgroups: comp.lang.c++,comp.lang.c
Subject: Re: Threads across programming languages
Date: Fri, 3 May 2024 10:34:11 +0200
Organization: A noiseless patient Spider
Lines: 61
Message-ID: <v127i4$ej9d$1@dont-email.me>
References: <GIL-20240429161553@ram.dialup.fu-berlin.de>
 <v0ogum$1rc5n$1@dont-email.me> <v0ovvl$1ur12$4@dont-email.me>
 <v0p06i$1uq6q$5@dont-email.me>
 <v0shti$2vrco$2@raubtier-asyl.eternal-september.org>
 <v0spsh$31ds4$3@dont-email.me>
 <v0stic$325kv$3@raubtier-asyl.eternal-september.org>
 <v0svtn$32o8h$1@dont-email.me>
 <v0t091$32qj6$1@raubtier-asyl.eternal-september.org>
 <v0u90h$3c1r5$4@dont-email.me> <v0v7rf$3lu04$1@dont-email.me>
 <v0v8u3$3m7rm$1@dont-email.me> <v10t0v$20cs$1@dont-email.me>
 <v116q4$4at1$1@dont-email.me> <v119bu$4pfa$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 03 May 2024 10:34:12 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="77e40aae3b22ade63afb79718be155c6";
	logging-data="478509"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18j/GkrQcwKb0PcD4FBloPT48Xn164lmk8="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Cancel-Lock: sha1:u8/Lc9pAQ5WIUU4uF4cvNnQFYzc=
Content-Language: en-GB
In-Reply-To: <v119bu$4pfa$1@dont-email.me>
Bytes: 4139

On 03/05/2024 01:58, Chris M. Thomasson wrote:
> On 5/2/2024 4:15 PM, Lawrence D'Oliveiro wrote:
>> On Thu, 2 May 2024 13:28:15 -0700, Chris M. Thomasson wrote:
>>
>>> On 5/1/2024 10:39 PM, Lawrence D'Oliveiro wrote:
>>>>
>>>> On Wed, 1 May 2024 22:20:47 -0700, Chris M. Thomasson wrote:
>>>>
>>>>> On 5/1/2024 1:34 PM, Lawrence D'Oliveiro wrote:
>>>>>
>>>>>> Remember, we’re talking about maximizing I/O throughput here, so CPU
>>>>>> is not the bottleneck.
>>>>>
>>>>> It can be if your thread synchronization scheme is sub par.
>>>>
>>>> Another reason to avoid threads.
>>>
>>> Why? Believe it or not, there are ways to create _highly_ scalable
>>> thread synchronization schemes.
>>
>> I’m sure there are. But none of that is relevant when the CPU isn’t the
>> bottleneck anyway.
> 
> The CPU can become a bottleneck. Depends on how the programmer 
> implements things.
> 
> 
>>>> So long as your async tasks have an await call somewhere in their main
>>>> loops, that should be sufficient to avoid most bottlenecks.
>>>
>>> async tasks are using threads... No?
>>
>> No. They are built on coroutines. Specifically, the “stackless” variety.
>>
>> <https://gitlab.com/ldo/python_topics_notebooks/-/blob/master/Generators%20&%20Coroutines.ipynb?ref_type=heads>
> 
> So, there is no way to take advantage of multiple threads on Python? 
> Heck, even JavaScript has WebWorkers... ;^)

Python supports multi-threading.  It uses a global lock (the "GIL") in 
the Python interpreter - thus only one thread can be running Python code 
at a time.  However, if you are doing anything serious with Python, much 
of the time will be spend either blocked (waiting for network, IO, etc.) 
or using compiled or external code (using your favourite gui toolkit, 
doing maths with numpy, etc.).  The GIL is released while executing such 
code.

Thus if you are using Python for cpu-intensive work (and doing so 
sensibly), you have full multi-threading.  If you are using it for 
IO-intensive work, you have full multi-threading.  It's not going to be 
as efficient as well-written compiled code, even with JIT and pypy, but 
in practice it gets pretty close while being very convenient and 
developer friendly.

If you really need parallel running of Python code, or better separation 
between tasks, Python has a multi-processing module that makes it simple 
to control and pass data between separate Python processes, each with 
their own GIL.