Path: ...!feeds.phibee-telecom.net!weretis.net!feeder8.news.weretis.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: MRAB <python@mrabarnett.plus.com>
Newsgroups: comp.lang.python
Subject: Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment
Date: Thu, 24 Oct 2024 20:54:53 +0100
Lines: 28
Message-ID: <mailman.40.1729799882.4695.python-list@python.org>
References: <4a13731716200669342338ae409e73ca@posteo.de>
 <CAJQBtg=kCr0VMGXvkBcj4U6CwatC2_G2dstO5pS56_koAXJurA@mail.gmail.com>
 <0c080a7d-92de-4518-ac44-78d97fc9e3bb@mrabarnett.plus.com>
 <CAJQBtg=ButLwvoA2USBEV8-q=taRhuC+t6D1HkJMbG9VzTvxnQ@mail.gmail.com>
 <69934a74-6d04-40a0-a75f-b8024bd0af43@mrabarnett.plus.com>
 <CAJQBtgm6Re8tj5hKAZ7Zz6+NC-0iqcBH-MKo8gaOMm8A4prZfA@mail.gmail.com>
 <db1bf48a-01a8-4f7e-8d13-33da26af7613@mrabarnett.plus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: news.uni-berlin.de RMZKsp5AB9ZaPVL9K4RUpwo4xlwWm23igWfQDui8tURA==
Cancel-Lock: sha1:ttp7u4AiunmLJf6dFljykPvSa/o= sha256:Th89BIC8X88Eedq21D/EIU9uZ0/3joV+6p2eW+9UZbQ=
Return-Path: <python@mrabarnett.plus.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
 reason="2048-bit key; unprotected key"
 header.d=plus.com header.i=@plus.com header.b=lrNXRW88;
 dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'thread': 0.05;
 'subject:when': 0.07; 'alternatives': 0.09; 'comparison': 0.09;
 'from:addr:python': 0.09; 'received:192.168.1.64': 0.09;
 'reference:': 0.09; 'resolved': 0.09; '651': 0.16; '677': 0.16;
 '679': 0.16; '689': 0.16; '760': 0.16; 'appended': 0.16; 'assert':
 0.16; 'comparison,': 0.16; 'failed.': 0.16;
 'from:addr:mrabarnett.plus.com': 0.16; 'from:name:mrab': 0.16;
 'lambda': 0.16; 'message-id:@mrabarnett.plus.com': 0.16;
 'received:plus.net': 0.16; 'subject:() ': 0.16; 'subject:Fedora':
 0.16; 'subject:fails': 0.16; 'subject:used': 0.16; 'wrote:': 0.16;
 "can't": 0.17; 'to:addr:python-list': 0.20; 'skip:_ 10': 0.22;
 'lines': 0.23; 'to:name:python-list@python.org': 0.24; 'object':
 0.26; 'else': 0.27; 'header:User-Agent:1': 0.30; 'modify': 0.31;
 'module': 0.31; 'question': 0.32; 'modified': 0.32; 'but': 0.32;
 'received:192.168.1': 0.32; 'header:In-Reply-To:1': 0.34;
 'following': 0.35; 'those': 0.36; 'received:192.168': 0.37; 'put':
 0.38; 'added': 0.39; 'something': 0.40; 'reference': 0.60;
 'between': 0.63; 'less': 0.64; 'probability': 0.64; 'that,': 0.68;
 'right': 0.68; 'skip:* 10': 0.73; 'left': 0.83; 'replacing': 0.91;
 'time?': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plus.com; s=042019;
 t=1729799694; bh=g/XAqRo6dJVzm5k2YXCK2sp7N8+VBwXieiMD6ZIC7Rk=;
 h=Date:Subject:To:References:From:In-Reply-To;
 b=lrNXRW88hNppAA/UEn3UN8PyQIStqsi+yfjVMwQ5qSJZ/Hqap9x33z4dHVwwF/Xn/
 Q7fugUCqL5yMHxt4grp4TiIdXZS1hBhmfkk4frT6WHQyUtkmqWF/DEy7u2GOtlwZO5
 A+90KGHmyao6GDT8xAOV6NCQ+zUnjPfYSBRHH55lclinMC54W0+fpxGI11Oc/3DNWy
 f+mKc6o8A/tIg+2JGJgKAYs7uoTC52Hua23fvLevNvvFiu7dTbmEgg3tEw8bJ5m32B
 6NFSC/XNZ8K1NAyT7ohh7bGMaPTE1dLxW+zrCI9PqE2S+c8oiDB3pidWLL4PRE64Oz
 W57nqb8C/8taw==
X-Clacks-Overhead: "GNU Terry Pratchett"
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.4 cv=JP6YsNKb c=1 sm=1 tr=0 ts=671aa60e
 a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17
 a=IkcTkHD0fZMA:10 a=GLbXuAWmxfJgwpdhh64A:9 a=QEXdDO2ut3YA:10
X-AUTH: mrabarnett@:2500
User-Agent: Mozilla Thunderbird
Content-Language: en-GB
In-Reply-To: <CAJQBtgm6Re8tj5hKAZ7Zz6+NC-0iqcBH-MKo8gaOMm8A4prZfA@mail.gmail.com>
X-CMAE-Envelope: MS4xfHsBJzupNfS/uGY3iQUZZBf8/4nXGARCC6lbX5BO/E8ls6vb/xhNmOy2ci6Coum5G4cpXtb//tvc3ed4ifjGq6qbHbZqdBI7a2yjv+4sCxlh29rE20/+
 6JJF5CBLlYmjduxSVFaYfkYdw6eDOdZ/IxI7WarNNloxYzByKK6++6t1i+IMG16k/MGeRTETr+7omNdUz9u3mjMNYsGvkPvOZng=
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
 <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
 <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
 <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <db1bf48a-01a8-4f7e-8d13-33da26af7613@mrabarnett.plus.com>
X-Mailman-Original-References: <4a13731716200669342338ae409e73ca@posteo.de>
 <CAJQBtg=kCr0VMGXvkBcj4U6CwatC2_G2dstO5pS56_koAXJurA@mail.gmail.com>
 <0c080a7d-92de-4518-ac44-78d97fc9e3bb@mrabarnett.plus.com>
 <CAJQBtg=ButLwvoA2USBEV8-q=taRhuC+t6D1HkJMbG9VzTvxnQ@mail.gmail.com>
 <69934a74-6d04-40a0-a75f-b8024bd0af43@mrabarnett.plus.com>
 <CAJQBtgm6Re8tj5hKAZ7Zz6+NC-0iqcBH-MKo8gaOMm8A4prZfA@mail.gmail.com>
Bytes: 6377

On 2024-10-24 20:21, Left Right wrote:
> > > > The stack is created on line 760 with os.lstat and entries are appended
> > > > on lines 677 (os.rmdir), 679 (os.close) and 689 (os.lstat).
> > > >
> > > > 'func' is popped off the stack on line 651 and check in the following lines.
> > > >
> > > > I can't see anywhere else where something else is put onto the stack or
> > > > an entry is replaced.
>
> But the _rmtree_safe_fd() compares func to a *dynamically* resolved
> reference: os.lstat. If the reference to os changed (or os object was
> modified to have new reference at lstat) between the time os.lstat was
> added to the stack and the time of comparison, then comparison
> would've failed.  To illustrate my idea:
>
> os.lstat = lambda x: x # thread 1
> stack.append((os.lstat, ...)) # thread 1
> os.lstat = lambda x: x # thread 2
> func, *_ = stack.pop() # thread 1
> assert func is os.lstat # thread 1 (failure!)
>
> The only question is: is it possible to modify os.lstat like that, and
> if so, how?
>
> Other alternatives include a malfunctioning "is" operator,
> malfunctioning module cache... all those are a lot less likely.
What is the probability of replacing os.lstat, os.close or os.rmdir from 
another thread at just the right time?