Path: ...!news.mixmin.net!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: Chris Angelico <rosuav@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Help with Streaming and Chunk Processing for Large JSON Data (60
 GB) from Kenna API
Date: Wed, 2 Oct 2024 23:59:41 +1000
Lines: 9
Message-ID: <mailman.28.1727877596.3018.python-list@python.org>
References: <CADrxXXmHUwsQbWqNrwzyKWLyTK0J3Hf0z8hAhGwKYoF2PwK7QA@mail.gmail.com>
 <082705B5-7C14-4D33-BF38-73F9CB166293@barrys-emacs.org>
 <9dfcd123-c31d-4207-869c-d5466487cba4@tompassin.net>
 <CAJQBtgkLVyNK+vw4u3bFCFEQDH8T3rpyTL+ERyyYHZJskQR6PQ@mail.gmail.com>
 <CAJQBtgnpNkpg-mF2yFCS4P4GYAYsKQ9nEw3Xygja=SE3-=N2Dw@mail.gmail.com>
 <mailman.19.1727796506.3018.python-list@python.org>
 <lm391bFu38hU1@mid.individual.net>
 <CAJQBtgmZehSeBu0y73ALdVq00LHi-R_KKS893FwJkEjkLnsXtA@mail.gmail.com>
 <CAPTjJmq6QUcBgkNcn50VzyyHoDAEE1JLPgPU+segiEykcieVSw@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de bpRCN3GtuwIr7/WGPIEoDAibA7VcQKTWKWhQyqmaOWhg==
Cancel-Lock: sha1:bYGTTeW94+QdUGHzvFi6N99/h/Y= sha256:aojQFJ4Yourih00fhb7xGiVj9GP27Ydeti2HButh2Vg=
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
 reason="2048-bit key; unprotected key"
 header.d=gmail.com header.i=@gmail.com header.b=JAafgSSu;
 dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.065
X-Spam-Evidence: '*H*': 0.87; '*S*': 0.00; 'subject:API': 0.07;
 '2024': 0.16; 'chrisa': 0.16; 'from:addr:rosuav': 0.16;
 'from:name:chris angelico': 0.16; 'received:mail-
 lj1-x235.google.com': 0.16; 'wrote:': 0.16; 'subject:Help': 0.17;
 "can't": 0.17; 'to:addr:python-list': 0.20; 'languages': 0.22;
 'examples': 0.25; 'python-list': 0.32; 'validate': 0.32; 'message-
 id:@mail.gmail.com': 0.32; 'subject:for': 0.33; 'same': 0.34;
 'header:In-Reply-To:1': 0.34; 'received:google.com': 0.34;
 'from:addr:gmail.com': 0.35; 'subject:from': 0.37; 'wed,': 0.39;
 'gave': 0.61; 'email': 0.63; 'your': 0.64; 'right': 0.68;
 'subject:Data': 0.71; 'left': 0.83; 'subject: \n ': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1727877594; x=1728482394; darn=python.org;
 h=to:subject:message-id:date:from:in-reply-to:references:mime-version
 :from:to:cc:subject:date:message-id:reply-to;
 bh=Y942I+l7q3jvGI1R+5cBfFq8tnQ8o6js80ejQMswFu0=;
 b=JAafgSSuWKtdzkwanyA/++iaOaE/7jWw578EZP7I7iV2IqYtJ5AQTvdsPqqYI5zYfC
 IxRWQBpzTSXEhHjj1iKTpuav1PvbmPljFS1APuwyWkg7Zce/S3jMWJIL4LuzCLQaZA4N
 q+mdhlpV1f7pJyZriH4Cabq6v2GqjPM2aDXH1a8D/nx8+L9NuQNZjcRgtM3K09TvmCj9
 7C8LQUyzQZmJY39imhnOfEX6ERlfyg8bAaBvWiGnFCPmqtFC3YrpvlkRsawrZrfxJe4B
 6ftI6FdBsMw3oKGc3ZjdaOwMbIN7+ywXkvqgkAQ3AhHvQk0DM8hoO3idMPcwo38h62fh
 z9LA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1727877594; x=1728482394;
 h=to:subject:message-id:date:from:in-reply-to:references:mime-version
 :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=Y942I+l7q3jvGI1R+5cBfFq8tnQ8o6js80ejQMswFu0=;
 b=TXFuNBYc2FITNLO/KBxjxA4RcsSiCtIrulHSEyhlY3ZTGrnDrUXkTea7kXlGegVq0A
 KHwoRtXuM+IiaHq6A4Mu/4heomK+kgsHxAyEQtYIdZp9hmbcENjDI+CeAx7E5UrWM+tf
 6F5Grq3hDocywVCYPHSo5wJ7IYu88HtW2fDL39Gu8/YveSVSJJ7/4bUf8CaLafkvlQ77
 /GVr5V2+oCRGScq3ZmrxwBhcfIJty3n8nh4Pnu6KrZde4yIYHVAHgVEb0Q63KYs0vs2N
 MMW9QbiJh80/gDjC2iXNvDSVY89n5y7HGj/FPToeqVY4nrb4J1pBtc8GhS9EdBqtzh+S
 pOow==
X-Gm-Message-State: AOJu0YwKKGZ8nkUfs3UXT7b9QCBgaSwZqtjfn/lU0wTtz3CxR1QJgCHh
 WGW53qIPPWTNGNwkVrTJDE7EgJlK6O94WG+VKvME0tLnAtKuVehoBKOkjEp9xg0E9AMJQposTYu
 Ch5XYEz1nEq2XPPpPuFPgn/fQ/FUt4w==
X-Google-Smtp-Source: AGHT+IFYW8B9dhSMCs6A2FnF9nsut2QRTChiVNlz7/qT7JffYMd+Za/yh9SclSH6uzdaP5qvBqdXlhaJG4cBCah0yYw=
X-Received: by 2002:a05:651c:2225:b0:2fa:d79d:d0e with SMTP id
 38308e7fff4ca-2fae102794fmr19044811fa.19.1727877593426; Wed, 02 Oct 2024
 06:59:53 -0700 (PDT)
In-Reply-To: <CAJQBtgmZehSeBu0y73ALdVq00LHi-R_KKS893FwJkEjkLnsXtA@mail.gmail.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
 <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
 <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
 <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAPTjJmq6QUcBgkNcn50VzyyHoDAEE1JLPgPU+segiEykcieVSw@mail.gmail.com>
X-Mailman-Original-References: <CADrxXXmHUwsQbWqNrwzyKWLyTK0J3Hf0z8hAhGwKYoF2PwK7QA@mail.gmail.com>
 <082705B5-7C14-4D33-BF38-73F9CB166293@barrys-emacs.org>
 <9dfcd123-c31d-4207-869c-d5466487cba4@tompassin.net>
 <CAJQBtgkLVyNK+vw4u3bFCFEQDH8T3rpyTL+ERyyYHZJskQR6PQ@mail.gmail.com>
 <CAJQBtgnpNkpg-mF2yFCS4P4GYAYsKQ9nEw3Xygja=SE3-=N2Dw@mail.gmail.com>
 <mailman.19.1727796506.3018.python-list@python.org>
 <lm391bFu38hU1@mid.individual.net>
 <CAJQBtgmZehSeBu0y73ALdVq00LHi-R_KKS893FwJkEjkLnsXtA@mail.gmail.com>
Bytes: 5699

On Wed, 2 Oct 2024 at 23:53, Left Right via Python-list
<python-list@python.org> wrote:
> In the same email you replied to, I gave examples of languages for
> which parsers can be streaming (in general): SCSI or IP.

You can't validate an IP packet without having all of it. Your notion
of "streaming" is nonsensical.

ChrisA