Deutsch   English   Français   Italiano  
<mailman.9.1720299957.2981.python-list@python.org>

View for Bookmarking (what is this?)
Look up another Usenet article

Path: ...!2.eu.feeder.erje.net!feeder.erje.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: dn <PythonList@DancesWithMice.info>
Newsgroups: comp.lang.python
Subject: Re: Best use of "open" context manager
Date: Sun, 7 Jul 2024 09:05:36 +1200
Organization: DWM
Lines: 106
Message-ID: <mailman.9.1720299957.2981.python-list@python.org>
References: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com>
 <cd084a1a-fe58-4190-92bb-6f4002517fa5@DancesWithMice.info>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de V+qyn25m6duP5i6dOOTLewiqgfKJNoelEVM3f1Kyg0bA==
Cancel-Lock: sha1:o0Ut03q51G1p4pAWcgIk1QUF9Wg= sha256:3JLs4FyPeTBi8a++U6z50Dv1O7elUSwWpBeyPaJKEOo=
Return-Path: <PythonList@DancesWithMice.info>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
 reason="2048-bit key; unprotected key"
 header.d=danceswithmice.info header.i=@danceswithmice.info
 header.b=gbrQsQu+; dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.064
X-Spam-Evidence: '*H*': 0.87; '*S*': 0.00; 'this:': 0.03; '(which':
 0.04; 'e.g.': 0.07; '=dn': 0.09; 'action,': 0.09; 'else:': 0.09;
 'from:addr:danceswithmice.info': 0.09; 'from:addr:pythonlist':
 0.09; 'memory.': 0.09; 'valueerror:': 0.09; 'coding': 0.13;
 'decreases': 0.16; 'ideal.': 0.16; 'message-
 id:@DancesWithMice.info': 0.16; 'oop': 0.16; 'pythonic': 0.16;
 'received:cloud': 0.16; 'received:rangi.cloud': 0.16; 'targeted':
 0.16; 'ways.': 0.16; 'whilst': 0.16; 'wrote:': 0.16; 'to:addr
 :python-list': 0.20; 'closed': 0.22; 'exception': 0.22; 'ran':
 0.22; 'code': 0.23; 'lines': 0.23; 'idea': 0.24; 'python,': 0.25;
 'seems': 0.26; 'manager,': 0.26; 'opening': 0.26; 'bit': 0.27;
 'function': 0.27; 'error': 0.29; 'header:User-Agent:1': 0.30;
 'attempt': 0.31; 'modify': 0.31; 'header:Organization:1': 0.31;
 'context': 0.32; 'python-list': 0.32; 'received:192.168.1': 0.32;
 'but': 0.32; 'there': 0.33; 'header:In-Reply-To:1': 0.34;
 'handling': 0.35; 'close': 0.35; 'work,': 0.36; 'really': 0.37;
 'using': 0.37; 'class': 0.37; 'received:192.168': 0.37; 'file':
 0.38; 'could': 0.38; 'put': 0.38; 'read': 0.38; 'two': 0.39;
 'quite': 0.39; 'added': 0.39; 'text': 0.39; 'program.': 0.40;
 'something': 0.40; 'want': 0.40; 'try': 0.40; 'should': 0.40;
 'kept': 0.61; 'remember': 0.61; 'skip:o 10': 0.61; "there's":
 0.61; 'here': 0.62; 'between': 0.63; 'similar': 0.65; 'less':
 0.65; 'per': 0.68; 'closing': 0.69; 'generator': 0.69; 'small,':
 0.69; 'within': 0.69; 'care': 0.71; 'suite': 0.71; 'relevant':
 0.73; 'easy': 0.74; 'handles': 0.76; 'exceptions': 0.84; 'forgot':
 0.84; 'habits': 0.84; 'readability.': 0.84; 'rob': 0.84; 'styles':
 0.84; 'subject:manager': 0.84; 'subject:open': 0.84; 'thus,':
 0.84; 'violates': 0.84; 'with...': 0.84; 'loses': 0.91; 'race':
 0.93
DKIM-Filter: OpenDKIM Filter v2.11.0 vps.rangi.cloud 2DECB41D0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=danceswithmice.info;
 s=staff; t=1720299942;
 bh=fJSmBkhRoKWrtMFQyQbslewpLSYe0jmL9yoVrAk3IOk=;
 h=Date:From:Subject:To:References:In-Reply-To:From;
 b=gbrQsQu+NI/3+KNf+k37t/d09lIHn5HyYmMWvtTFvQiP7zECH1Jjg8BoUxK8/qn/V
 FuK6BQv/FQYPjhtvHsE4cdDyqME1340i3+LmGUtVQC9aeQxJ4pAEK4R+Rph18vj78w
 aD7nh4ozrNVsfPOUjBESCb1fOlogWW/vUBbTUaYDdIejAUurxbH0Cp13dJU/bR9D5p
 aPz3h3/1bJZgsUnWJNRMUNo8ddBK5ieFm1Cgx4lxIA58qodazduXCB/B42s7RwEgm5
 Jg0RmszBDYLxnK8gFp81wmu3XepzsGm0oqmh9Asma1TZDAMxt1BBkR+vwf1cMvMlss
 ZMi44UqxihoZw==
User-Agent: Mozilla Thunderbird
Content-Language: en-US
In-Reply-To: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
 <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
 <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
 <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <cd084a1a-fe58-4190-92bb-6f4002517fa5@DancesWithMice.info>
X-Mailman-Original-References: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com>
Bytes: 8119

On 6/07/24 22:49, Rob Cliffe via Python-list wrote:
> Consider this scenario (which I ran into in real life):
>      I want to open a text file and do a lot of processing on the lines 
> of that file.
>      If the file does not exist I want to take appropriate action, e.g. 
> print an error message and abort the program.
> I might write it like this:
> 
> try:
>      with open(FileName) as f:
>          for ln in f:
>              print("I do a lot of processing here")
>              # Many lines of code here .....
> except FileNotFoundError:
>      print(f"File {FileName} not found")
>      sys.exit()
> 
> but this violates the principle that a "try" suite should be kept small, 
> so that only targeted exceptions are trapped,

Yes!


> not to mention that having "try" and "except" far apart decreases 
> readability.

Uh-oh!

- and there's a bit of a hang-over for old-timers who had to take care 
of file-locking within the application - we try to minimise 'time' 
between opening a file and closing it (etc)!

As it seems the file is opened to read. Less relevant in this case, but 
habits and styles of coding matter...


> Or I might write it like this:
> 
> try:
>      f = open(FileName) as f:
>      FileLines = f.readlines()
> except FileNotFoundError:
>      print(f"File {FileName} not found")
>      sys.exit()
> # I forgot to put "f.close()" here -:)
> for ln in File Lines:
>          print("I do a lot of processing here")
>          # Many lines of code here .....
> 
> but this loses the benefits of using "open" as a context manager,
> and would also be unacceptable if the file was too large to read into 
> memory.

So, now there are two concerns:
1 FileNotFoundError, and
2 gradual processing to avoid memory-full

- added to remembering to close the file.


> Really I would like to write something like
> 
> try:
>      with open(FileName) as f:
> except FileNotFoundError:
>      print(f"File {FileName} not found")
>      sys.exit()
> else: # or "finally:"
>          for ln in f:
>              print("I do a lot of processing here")
>              # Many lines of code here .....
> 
> but this of course does not work because by the time we get to "for ln 
> in f:" the file has been closed so we get
> ValueError: I/O operation on closed file
> 
> I could modify the last attempt to open the file twice, which would 
> work, but seems like a kludge (subject to race condition, inefficient).
> 
> Is there a better / more Pythonic solution?

Idea 1: invert the exception handling and the context-manager by writing 
a custom context-manager class which handles FileNotFoundError 
internally. Thus, calling-code becomes:

with...
     for...
         processing

Idea 2: incorporate idea of encapsulating "processing" into a 
(well-named) function to shorten the number of lines-of-code inside the 
with-suite.

Idea 3: consider using a generator to 'produce' lines of data 
one-at-a-time. Remember that whilst context-managers and generators are 
distinct concepts within Python, they are quite similar in many ways. 
So, a custom generator could work like a context-manager or 'wrap' a 
context-manager per Idea 1.

Building a custom-class (Idea 1 or Idea 3) enables the components to be 
kept together, per the ideal. It keeps the try-except components close 
and easy to relate. It is Pythonic (in the OOP style).

-- 
Regards,
=dn