Path: ...!fu-berlin.de!uni-berlin.de!not-for-mail
From: Chris Angelico <rosuav@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Help with Streaming and Chunk Processing for Large JSON Data (60
 GB) from Kenna API
Date: Tue, 1 Oct 2024 09:09:07 +1000
Lines: 21
Message-ID: <mailman.17.1727737763.3018.python-list@python.org>
References: <CA+hg4RiGjXw3am1s=zVLDpcA-VGS+cWNp_YEyzvS+j2MyDE2Cg@mail.gmail.com>
 <CADrxXXmHUwsQbWqNrwzyKWLyTK0J3Hf0z8hAhGwKYoF2PwK7QA@mail.gmail.com>
 <CA+hg4Rhn8iX7rp0uC=MbOi+8g73wQ4y4=uV0dU0jHdDUz3jk4w@mail.gmail.com>
 <CAJQBtgk122sHzs+=MumYM1HW2DwKm1+i02bqgBKh4oUJYievCg@mail.gmail.com>
 <4XHQPG4LzsznVwM@mail.python.org> <Zvrt0RJe5omaFkQq@anomaly>
 <CAPTjJmqCz0UthKfs2-sd6E0Jcq23m0r-DgriywwazhWZ381wwg@mail.gmail.com>
 <ZvsjI-E8Qtz4rCeL@anomaly> <4XHbxS5jl4znVGD@mail.python.org>
 <CAPTjJmr61ZfAHEUh6NF=EnyLHOTvm_qFrCb75n42Cq_cGdaTWQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de rSoqlZGSL/8/p6YDRt8daQGu/LqsAfXcBTMUmj3duvYA==
Cancel-Lock: sha1:JwCoFmTs4Y7We+aHdNlpZod110g= sha256:JKyaUU4inp9CIt6oP1UAbTniXCQAydnuGN7oKwiSnOE=
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
 reason="2048-bit key; unprotected key"
 header.d=gmail.com header.i=@gmail.com header.b=LPt9dx/s;
 dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.018
X-Spam-Evidence: '*H*': 0.96; '*S*': 0.00; 'subject:API': 0.07; 'dan':
 0.09; '2024': 0.16; 'anyway.': 0.16; 'base.': 0.16; 'chrisa':
 0.16; 'explanatory': 0.16; 'from:addr:rosuav': 0.16;
 'from:name:chris angelico': 0.16; 'integer': 0.16; 'odd': 0.16;
 'reason.': 0.16; 'whatever.': 0.16; 'wrote:': 0.16;
 'subject:Help': 0.17; 'grant': 0.17; 'tue,': 0.19; 'to:addr
 :python-list': 0.20; 'written': 0.22; 'notes': 0.26; 'sense':
 0.28; 'think': 0.32; "doesn't": 0.32; 'python-list': 0.32;
 'message-id:@mail.gmail.com': 0.32; "i'm": 0.33; 'subject:for':
 0.33; 'header:In-Reply-To:1': 0.34; 'received:google.com': 0.34;
 'trying': 0.35; 'from:addr:gmail.com': 0.35; 'people': 0.36;
 'subject:from': 0.37; "it's": 0.37; 'hard': 0.37; 'two': 0.39;
 'base': 0.40; 'tell': 0.60; 'clear': 0.64; 'numbers': 0.67;
 'knowing': 0.71; 'subject:Data': 0.71; 'identifies': 0.84; 'six,':
 0.84; 'subject: \n ': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1727737760; x=1728342560; darn=python.org;
 h=to:subject:message-id:date:from:in-reply-to:references:mime-version
 :from:to:cc:subject:date:message-id:reply-to;
 bh=ceG2Y+aEpUiNcad8qikbuY0V/kuzq7p/ywlK+r2OLPQ=;
 b=LPt9dx/seOj3ON8txY9WOuVZEyJGvBxKvskMkXhL4Up1RpaQoFboMJf9QFXLJcm6fq
 myWeYlZm/d8Gb0nI7ObhmsBZYS4hxoRiCNcwE5jSsRtuDk2Kxeir7WPpORHdXjGk31KW
 VqJp+75fppe2VVmLr2MsB9tX0IACBAEdFiO9AUO3pvTCuRqSp6ImB/h0s76f861HkC/D
 488ZVliTM+GyIauEIkjGbo+thJOTR4P7PKBXMpTpe26hVTTYaWNw6S5U4MD17ZmyMD3X
 j6x2k2ontCxGG7mXRWPJvGdZbbr0wDh/WHsKDQQ6Bl4c3xDzEMQc9JLlA/9bUnY4xPsG
 9oBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1727737760; x=1728342560;
 h=to:subject:message-id:date:from:in-reply-to:references:mime-version
 :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=ceG2Y+aEpUiNcad8qikbuY0V/kuzq7p/ywlK+r2OLPQ=;
 b=Wz3iBcVBANAG4O731ty5XsEFizwdo5KEq3nJ5y3tN31Gy6OMmPPM4d8lg/r21ky/Df
 fUn9oqq3x0M6Mpv5lxBPeZOJIjuraCDxPhk+HvBDRuBVxP2do9pDgT3FeD6mI3RyAxKE
 jU+QN0WgNc8/eWRg8OE33K5+yaIkir97Ovx3S5pxl+3bw9eC3MiVMjIe2Z24C9dAf547
 pCTBJlvlVAGa9GqUrGJNI6IeTx0EsADCYJhOW5yGlwgj7aEX0KoOWxf4c8hxMiOn1u76
 uKssNNwdVJLasjl/ERkBS87IirttQmyg/DuK/xlKtIicgFaR4u/5Bq5aaI4z4jGAYLVo
 sm8w==
X-Gm-Message-State: AOJu0Yy6PtoMdjCD6lWP7kzk+XXnICfrwdKcl6rXyz/4eAo58rBWWGlO
 wyBDA0qmLt6j8ZDYds7h5f51Wily6kegC1I4Yb7n6JeqBG1vdEmS0yB+OjF8pI/W55s0xHOHxPc
 EBQkfHVn52935rcGDVJyPRjaTeMEtFQ==
X-Google-Smtp-Source: AGHT+IGSBW8ACeXi4Le3m2ek8hLi95uCFsroXou3YoeEq7K1GfdrROLVmTm/cPwnNICOJsw36/rnyzlmUOzxPjVPifU=
X-Received: by 2002:a05:6512:3e0d:b0:539:93b2:1380 with SMTP id
 2adb3069b0e04-53993b21678mr3792836e87.48.1727737760334; Mon, 30 Sep 2024
 16:09:20 -0700 (PDT)
In-Reply-To: <4XHbxS5jl4znVGD@mail.python.org>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
 <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
 <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
 <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAPTjJmr61ZfAHEUh6NF=EnyLHOTvm_qFrCb75n42Cq_cGdaTWQ@mail.gmail.com>
X-Mailman-Original-References: <CA+hg4RiGjXw3am1s=zVLDpcA-VGS+cWNp_YEyzvS+j2MyDE2Cg@mail.gmail.com>
 <CADrxXXmHUwsQbWqNrwzyKWLyTK0J3Hf0z8hAhGwKYoF2PwK7QA@mail.gmail.com>
 <CA+hg4Rhn8iX7rp0uC=MbOi+8g73wQ4y4=uV0dU0jHdDUz3jk4w@mail.gmail.com>
 <CAJQBtgk122sHzs+=MumYM1HW2DwKm1+i02bqgBKh4oUJYievCg@mail.gmail.com>
 <4XHQPG4LzsznVwM@mail.python.org> <Zvrt0RJe5omaFkQq@anomaly>
 <CAPTjJmqCz0UthKfs2-sd6E0Jcq23m0r-DgriywwazhWZ381wwg@mail.gmail.com>
 <ZvsjI-E8Qtz4rCeL@anomaly> <4XHbxS5jl4znVGD@mail.python.org>
Bytes: 6401

On Tue, 1 Oct 2024 at 08:56, Grant Edwards via Python-list
<python-list@python.org> wrote:
>
> On 2024-09-30, Dan Sommers via Python-list <python-list@python.org> wrote:
>
> > In Common Lisp, integers can be written in any integer base from two
> > to thirty six, inclusive.  So knowing the last digit doesn't tell
> > you whether an integer is even or odd until you know the base
> > anyway.
>
> I had to think about that for an embarassingly long time before it
> clicked.

The only part I'm not clear on is what identifies the base. If you're
going to write numbers little-endian, it's not that hard to also write
them with a base indicator before the digits. But, whatever. This is a
typical tangent and people are argumentative for no reason. I was just
trying to add some explanatory notes to why little-endian does make
more sense than big-endian.

ChrisA