Path: ...!fu-berlin.de!uni-berlin.de!not-for-mail
From: Tomas Ukkonen <tomas.ukkonen@protonmail.ch>
Newsgroups: comp.lang.python
Subject: Re: PyGILState_Ensure() deadlocks, why?
Date: Sun, 07 Jul 2024 19:40:56 +0000
Lines: 199
Message-ID: <mailman.12.1720386403.2981.python-list@python.org>
References: <y-6AMIAbWkvLII_tfcbRs5NtbKgQRos3XlZ83qbGCwuq7RSQJBGniGJEn4Nzwr_iVQfJOdvLVzOycTR0V-F-pDKt2rNPj3cHmFVY06vNQnI=@protonmail.ch>
 <DGC1c0YSrMcV0n6rNNeplHySXzBWUfTUwJbBVPgOocO1evufk5MKe8QWWEzO6Zj5-_0VrJJbK3OpB5-vrvMdDRKITiZrvq0RMY69IND05u4=@protonmail.ch>
Mime-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
 micalg=pgp-sha256;
 boundary="------cea9a307b89f84c86fe7aee189ffbbfe37df1fbb2efb3b4f35743011c9dd2ff9";
 charset=utf-8
X-Trace: news.uni-berlin.de dOqp9OQ8Adst2qNbpjWwYwHDfx+XO6KmPhAtY1bZyXXw==
Cancel-Lock: sha1:yf59Aw2ZaitC7tSo4fOMSNXAZI8= sha256:RoS8jB9TK2e0tyApWmdZN7TxuP8ZAH1L9yqhryTPJG0=
Return-Path: <tomas.ukkonen@protonmail.ch>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
 reason="2048-bit key; unprotected key"
 header.d=protonmail.ch header.i=@protonmail.ch header.b=FhRvtV7a;
 dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'content-
 type:multipart/signed': 0.05; 'thread': 0.05; 'content-
 type:application/pgp-signature': 0.09; 'filename:fname piece:asc':
 0.09; 'filename:fname piece:signature': 0.09;
 'filename:fname:signature.asc': 0.09; 'int': 0.09; 'linux': 0.09;
 'skip:` 10': 0.09; 'subject:skip:P 10': 0.09; 'threads': 0.09;
 'ubuntu': 0.09; '#include': 0.16; '<': 0.16; '2024': 0.16;
 'c++': 0.16; 'content-disposition:attachment;
 filename="signature.asc"': 0.16; 'gil': 0.16; 'gnu/linux': 0.16;
 'initialize': 0.16; 'interpreter': 0.16; 'long-running': 0.16;
 'main()': 0.16; 'subject:() ': 0.16; 'vector': 0.16; 'void': 0.16;
 'python': 0.16; 'api': 0.17; 'larger': 0.17; 'code.': 0.17; 'bug':
 0.19; 'to:addr:python-list': 0.20; 'code': 0.23; 'skip:p 30':
 0.23; 'to:name:python-list@python.org': 0.24; 'received:185.70':
 0.26; 'received:protonmail.ch': 0.26; 'task': 0.26; 'there': 0.33;
 'release': 0.34; 'header:In-Reply-To:1': 0.34; 'received:ch':
 0.35; 'following': 0.35; 'still': 0.40; 'skip:t 40': 0.64;
 'content-type:multipart/mixed': 0.68; 'received:185': 0.71;
 'header:Received:2': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.ch;
 s=protonmail3; t=1720381261; x=1720640461;
 bh=rBNP8/EtFG5rzRY4Gu9LMbbBrYXijQ4acdRuG6Mkk/U=;
 h=Date:To:From:Subject:Message-ID:In-Reply-To:References:
 Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
 Message-ID:BIMI-Selector;
 b=FhRvtV7a17sivB+yks/JUfjIXBMsfAWWMl2N4UKk7jhhNUEm2Ba4DOHzqk8U2BVb3
 c0AyPTcohc2699LVzYHNOwXKdnox3pOP/iIuSZPrPDBiLy6oHQT7TTMJeH3QwnvouC
 81SFVBUCsQSlq/tVnP2I1Qnn7zDdezOukR9uA4GynftH418JywU3Xh/dSql43X8lBj
 uZEOLk+BeoaUxTUpxf1sM3BEg47W3vJPNkJ3uqL1CMG74idUuoC0lAeWrIOoey4Y8Y
 JNmAM/+a4Wt7vPNi15kZExH0rs+BdTLU/grQpdp5bcuMU4WFB473EgBcoJfZBSxSrq
 lmlEZdbjk0qpQ==
In-Reply-To: <y-6AMIAbWkvLII_tfcbRs5NtbKgQRos3XlZ83qbGCwuq7RSQJBGniGJEn4Nzwr_iVQfJOdvLVzOycTR0V-F-pDKt2rNPj3cHmFVY06vNQnI=@protonmail.ch>
Feedback-ID: 809323:user:proton
X-Pm-Message-ID: 898b2bce32b2a34cfd43ca5e0be44b06d75df575
X-Mailman-Approved-At: Sun, 07 Jul 2024 17:06:42 -0400
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
 <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
 <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
 <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <DGC1c0YSrMcV0n6rNNeplHySXzBWUfTUwJbBVPgOocO1evufk5MKe8QWWEzO6Zj5-_0VrJJbK3OpB5-vrvMdDRKITiZrvq0RMY69IND05u4=@protonmail.ch>
X-Mailman-Original-References: <y-6AMIAbWkvLII_tfcbRs5NtbKgQRos3XlZ83qbGCwuq7RSQJBGniGJEn4Nzwr_iVQfJOdvLVzOycTR0V-F-pDKt2rNPj3cHmFVY06vNQnI=@protonmail.ch>
Bytes: 9534

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------cea9a307b89f84c86fe7aee189ffbbfe37df1fbb2efb3b4f35743011c9dd2ff9
Content-Type: multipart/mixed;boundary=---------------------9e29034e00ea96b7d10a477458ea4ed4

-----------------------9e29034e00ea96b7d10a477458ea4ed4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;charset=utf-8

Hi

There was a bug in the example code. I fixed it and it STILL deadlocks (my=
 larger software project deadlocks when I call python from C++).

Updated code:

/*=C2=A0* This code deadlocks on python3-dev 3.12.3 (ubuntu 24.04 lts)
=C2=A0*
=C2=A0* g++ python_thread_test.cpp `python3-config --cflags --libs --embed=
`
=C2=A0* ./a.out
=C2=A0*
=C2=A0* uname:
=C2=A0* Linux softice 6.8.0-36-generic SMP PREEMPT_DYNAMIC x86_64 GNU/Linu=
x
=C2=A0*/

#include <Python.h>
#include <thread>
#include <vector>
#include <iostream>


void thread_function() {
=C2=A0 =C2=A0 // Ensure this thread has the GIL
=C2=A0 =C2=A0 PyGILState_STATE gstate =3D PyGILState_Ensure();

=C2=A0 =C2=A0 // Execute some Python code
=C2=A0 =C2=A0 PyRun_SimpleString("print('Hello from std::thread!')");

=C2=A0 =C2=A0 // Release the GIL
=C2=A0 =C2=A0 PyGILState_Release(gstate);
}

int main() {
=C2=A0 =C2=A0 // Initialize the Python Interpreter
=C2=A0 =C2=A0 Py_Initialize();

=C2=A0 =C2=A0 // Create a vector of threads
=C2=A0 =C2=A0 std::vector<std::thread> threads;

=C2=A0 =C2=A0 // Launch threads
=C2=A0 =C2=A0 for (int i =3D 0; i < 5; ++i) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 threads.push_back(std::thread(thread_function)=
);
=C2=A0 =C2=A0 }

=C2=A0 =C2=A0 // Join threads
=C2=A0 =C2=A0 for (auto& t : threads) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 t.join();
=C2=A0 =C2=A0 }

=C2=A0 =C2=A0 // Finalize the Python Interpreter
=C2=A0 =C2=A0 Py_Finalize();

=C2=A0 =C2=A0 return 0;
}
sunnuntaina 7. hein=C3=A4kuuta 2024 klo 10:24 ip, Tomas Ukkonen <tomas.ukk=
onen@protonmail.ch> kirjoitti:

> Hello
> Is this python c api bug? The following C++ code (g++) deadlocks on Ubun=
tu Linux.
> =


> /*=C2=A0* This code deadlocks on python3-dev 3.12.3 (ubuntu 24.04 lts)
> =C2=A0*
> =C2=A0* g++ python_thread_test.cpp `python3-config --cflags --libs --emb=
ed`
> =C2=A0* ./a.out
> =C2=A0*
> =C2=A0* uname:
> =C2=A0* Linux softice 6.8.0-36-generic SMP PREEMPT_DYNAMIC x86_64 GNU/Li=
nux
> =C2=A0*/
> =


> #include <Python.h>
> #include <thread>
> #include <vector>
> #include <iostream>
> =


> void perform_long_operation() {
> =C2=A0 =C2=A0 // Simulate a long-running task
> =C2=A0 =C2=A0 std::this_thread::sleep_for(std::chrono::seconds(5));
> }
> =


> void thread_function() {
> =C2=A0 =C2=A0 // Ensure this thread has the GIL
> =C2=A0 =C2=A0 PyGILState_STATE gstate =3D PyGILState_Ensure();
> =


> =C2=A0 =C2=A0 // Execute some Python code
> =C2=A0 =C2=A0 PyRun_SimpleString("print('Hello from std::thread!')");
> =


> =C2=A0 =C2=A0 // Release the GIL for long operation
> =C2=A0 =C2=A0 Py_BEGIN_ALLOW_THREADS
> =C2=A0 =C2=A0 perform_long_operation();
> =C2=A0 =C2=A0 Py_END_ALLOW_THREADS
> =


> =C2=A0 =C2=A0 // Re-acquire the GIL and execute more Python code
> =C2=A0 =C2=A0 gstate =3D PyGILState_Ensure();
> =C2=A0 =C2=A0 PyRun_SimpleString("print('Thread operation completed!')")=
;
========== REMAINDER OF ARTICLE TRUNCATED ==========