Deutsch English Français Italiano |
<mailman.9.1720299957.2981.python-list@python.org> View for Bookmarking (what is this?) Look up another Usenet article |
Path: ...!2.eu.feeder.erje.net!feeder.erje.net!fu-berlin.de!uni-berlin.de!not-for-mail From: dn <PythonList@DancesWithMice.info> Newsgroups: comp.lang.python Subject: Re: Best use of "open" context manager Date: Sun, 7 Jul 2024 09:05:36 +1200 Organization: DWM Lines: 106 Message-ID: <mailman.9.1720299957.2981.python-list@python.org> References: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com> <cd084a1a-fe58-4190-92bb-6f4002517fa5@DancesWithMice.info> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: news.uni-berlin.de V+qyn25m6duP5i6dOOTLewiqgfKJNoelEVM3f1Kyg0bA== Cancel-Lock: sha1:o0Ut03q51G1p4pAWcgIk1QUF9Wg= sha256:3JLs4FyPeTBi8a++U6z50Dv1O7elUSwWpBeyPaJKEOo= Return-Path: <PythonList@DancesWithMice.info> X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org Authentication-Results: mail.python.org; dkim=pass reason="2048-bit key; unprotected key" header.d=danceswithmice.info header.i=@danceswithmice.info header.b=gbrQsQu+; dkim-adsp=pass; dkim-atps=neutral X-Spam-Status: OK 0.064 X-Spam-Evidence: '*H*': 0.87; '*S*': 0.00; 'this:': 0.03; '(which': 0.04; 'e.g.': 0.07; '=dn': 0.09; 'action,': 0.09; 'else:': 0.09; 'from:addr:danceswithmice.info': 0.09; 'from:addr:pythonlist': 0.09; 'memory.': 0.09; 'valueerror:': 0.09; 'coding': 0.13; 'decreases': 0.16; 'ideal.': 0.16; 'message- id:@DancesWithMice.info': 0.16; 'oop': 0.16; 'pythonic': 0.16; 'received:cloud': 0.16; 'received:rangi.cloud': 0.16; 'targeted': 0.16; 'ways.': 0.16; 'whilst': 0.16; 'wrote:': 0.16; 'to:addr :python-list': 0.20; 'closed': 0.22; 'exception': 0.22; 'ran': 0.22; 'code': 0.23; 'lines': 0.23; 'idea': 0.24; 'python,': 0.25; 'seems': 0.26; 'manager,': 0.26; 'opening': 0.26; 'bit': 0.27; 'function': 0.27; 'error': 0.29; 'header:User-Agent:1': 0.30; 'attempt': 0.31; 'modify': 0.31; 'header:Organization:1': 0.31; 'context': 0.32; 'python-list': 0.32; 'received:192.168.1': 0.32; 'but': 0.32; 'there': 0.33; 'header:In-Reply-To:1': 0.34; 'handling': 0.35; 'close': 0.35; 'work,': 0.36; 'really': 0.37; 'using': 0.37; 'class': 0.37; 'received:192.168': 0.37; 'file': 0.38; 'could': 0.38; 'put': 0.38; 'read': 0.38; 'two': 0.39; 'quite': 0.39; 'added': 0.39; 'text': 0.39; 'program.': 0.40; 'something': 0.40; 'want': 0.40; 'try': 0.40; 'should': 0.40; 'kept': 0.61; 'remember': 0.61; 'skip:o 10': 0.61; "there's": 0.61; 'here': 0.62; 'between': 0.63; 'similar': 0.65; 'less': 0.65; 'per': 0.68; 'closing': 0.69; 'generator': 0.69; 'small,': 0.69; 'within': 0.69; 'care': 0.71; 'suite': 0.71; 'relevant': 0.73; 'easy': 0.74; 'handles': 0.76; 'exceptions': 0.84; 'forgot': 0.84; 'habits': 0.84; 'readability.': 0.84; 'rob': 0.84; 'styles': 0.84; 'subject:manager': 0.84; 'subject:open': 0.84; 'thus,': 0.84; 'violates': 0.84; 'with...': 0.84; 'loses': 0.91; 'race': 0.93 DKIM-Filter: OpenDKIM Filter v2.11.0 vps.rangi.cloud 2DECB41D0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=danceswithmice.info; s=staff; t=1720299942; bh=fJSmBkhRoKWrtMFQyQbslewpLSYe0jmL9yoVrAk3IOk=; h=Date:From:Subject:To:References:In-Reply-To:From; b=gbrQsQu+NI/3+KNf+k37t/d09lIHn5HyYmMWvtTFvQiP7zECH1Jjg8BoUxK8/qn/V FuK6BQv/FQYPjhtvHsE4cdDyqME1340i3+LmGUtVQC9aeQxJ4pAEK4R+Rph18vj78w aD7nh4ozrNVsfPOUjBESCb1fOlogWW/vUBbTUaYDdIejAUurxbH0Cp13dJU/bR9D5p aPz3h3/1bJZgsUnWJNRMUNo8ddBK5ieFm1Cgx4lxIA58qodazduXCB/B42s7RwEgm5 Jg0RmszBDYLxnK8gFp81wmu3XepzsGm0oqmh9Asma1TZDAMxt1BBkR+vwf1cMvMlss ZMi44UqxihoZw== User-Agent: Mozilla Thunderbird Content-Language: en-US In-Reply-To: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com> X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: General discussion list for the Python programming language <python-list.python.org> List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe> List-Archive: <https://mail.python.org/pipermail/python-list/> List-Post: <mailto:python-list@python.org> List-Help: <mailto:python-list-request@python.org?subject=help> List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe> X-Mailman-Original-Message-ID: <cd084a1a-fe58-4190-92bb-6f4002517fa5@DancesWithMice.info> X-Mailman-Original-References: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com> Bytes: 8119 On 6/07/24 22:49, Rob Cliffe via Python-list wrote: > Consider this scenario (which I ran into in real life): > I want to open a text file and do a lot of processing on the lines > of that file. > If the file does not exist I want to take appropriate action, e.g. > print an error message and abort the program. > I might write it like this: > > try: > with open(FileName) as f: > for ln in f: > print("I do a lot of processing here") > # Many lines of code here ..... > except FileNotFoundError: > print(f"File {FileName} not found") > sys.exit() > > but this violates the principle that a "try" suite should be kept small, > so that only targeted exceptions are trapped, Yes! > not to mention that having "try" and "except" far apart decreases > readability. Uh-oh! - and there's a bit of a hang-over for old-timers who had to take care of file-locking within the application - we try to minimise 'time' between opening a file and closing it (etc)! As it seems the file is opened to read. Less relevant in this case, but habits and styles of coding matter... > Or I might write it like this: > > try: > f = open(FileName) as f: > FileLines = f.readlines() > except FileNotFoundError: > print(f"File {FileName} not found") > sys.exit() > # I forgot to put "f.close()" here -:) > for ln in File Lines: > print("I do a lot of processing here") > # Many lines of code here ..... > > but this loses the benefits of using "open" as a context manager, > and would also be unacceptable if the file was too large to read into > memory. So, now there are two concerns: 1 FileNotFoundError, and 2 gradual processing to avoid memory-full - added to remembering to close the file. > Really I would like to write something like > > try: > with open(FileName) as f: > except FileNotFoundError: > print(f"File {FileName} not found") > sys.exit() > else: # or "finally:" > for ln in f: > print("I do a lot of processing here") > # Many lines of code here ..... > > but this of course does not work because by the time we get to "for ln > in f:" the file has been closed so we get > ValueError: I/O operation on closed file > > I could modify the last attempt to open the file twice, which would > work, but seems like a kludge (subject to race condition, inefficient). > > Is there a better / more Pythonic solution? Idea 1: invert the exception handling and the context-manager by writing a custom context-manager class which handles FileNotFoundError internally. Thus, calling-code becomes: with... for... processing Idea 2: incorporate idea of encapsulating "processing" into a (well-named) function to shorten the number of lines-of-code inside the with-suite. Idea 3: consider using a generator to 'produce' lines of data one-at-a-time. Remember that whilst context-managers and generators are distinct concepts within Python, they are quite similar in many ways. So, a custom generator could work like a context-manager or 'wrap' a context-manager per Idea 1. Building a custom-class (Idea 1 or Idea 3) enables the components to be kept together, per the ideal. It keeps the try-except components close and easy to relate. It is Pythonic (in the OOP style). -- Regards, =dn