Deutsch English Français Italiano |
<GIL-20240429161553@ram.dialup.fu-berlin.de> View for Bookmarking (what is this?) Look up another Usenet article |
Path: ...!fu-berlin.de!uni-berlin.de!not-for-mail From: ram@zedat.fu-berlin.de (Stefan Ram) Newsgroups: comp.lang.c++,comp.lang.c Subject: Threads across programming languages Date: 29 Apr 2024 15:19:19 GMT Organization: Stefan Ram Lines: 46 Expires: 1 Feb 2025 11:59:58 GMT Message-ID: <GIL-20240429161553@ram.dialup.fu-berlin.de> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Trace: news.uni-berlin.de kdNvl5nF6zF8DmBi02H4aggNAdqfup4ILd/e1LEZpMb7+x Cancel-Lock: sha1:zDUIv81SWBYBWs9/mTGbTkQA9/k= sha256:+5HVd02oA4hgHbUCCuUkJiQ1KATTiW38xnufcHjiNuI= X-Copyright: (C) Copyright 2024 Stefan Ram. All rights reserved. Distribution through any means other than regular usenet channels is forbidden. It is forbidden to publish this article in the Web, to change URIs of this article into links, and to transfer the body without this notice, but quotations of parts in other Usenet posts are allowed. X-No-Archive: Yes Archive: no X-No-Archive-Readme: "X-No-Archive" is set, because this prevents some services to mirror the article in the web. But the article may be kept on a Usenet archive server with only NNTP access. X-No-Html: yes Content-Language: en-US Bytes: 3557 paavo512 <paavo@osa.pri.ee> wrote or quoted: |Anyway, multithreading performance is a non-issue for Python so far as |the Python interpreter runs in a single-threaded regime anyway, under a |global GIL lock. They are planning to get rid of GIL, but this work is |still in development AFAIK. I'm sure it will take years to stabilize the |whole Python zoo without GIL. The GIL only prevents multiple Python statements from being interpreted simultaneously, but if you're waiting on inputs (like sockets), it's not active, so that could be distributed across multiple cores. With asyncio, however, you can easily handle the application for threads to "wait in parallel" for thousands of sockets in a single thread, and there are fewer opportunities for errors than with multithreading. Additionally, there are libraries like numpy that use true multithreading internally to distribute computational tasks across multiple cores. By using such libraries, you can take advantage of that. (Not to mention the AI libraries that have their work done in highly parallel fashion by graphics cards.) If you want real threads, you could probably work with Cython sometimes. Other languages like JavaScript seem to have an advantage there because they don't know a GIL, but with JavaScript, for example, it's because it always runs in a single thread overall. And in the languages where there are threads without a GIL, you quickly realize that programming correct non-trivial programs with parallel processing is error-prone. Often in Python you can use "ThreadPoolExecutor" to start multiple threads. If the GIL then becomes a problem (which is not the case if you're waiting on I/O), you can easily swap it out for "ProcessPoolExecutor": Then processes are used instead of threads, and there is no GIL for those. If four cores are available, by dividing up compute-intensive tasks using "ProcessPoolExecutor", you can expect a speedup factor of two to eight. With the Celery library, tasks can be distributed across multiple processes that can also run on different computers. See, for example, "Parallel Programming with Python" by Jan Palach.