Path: ...!weretis.net!feeder6.news.weretis.net!feeder8.news.weretis.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: immibis <news@immibis.com>
Newsgroups: news.admin.peering,news.software.nntp
Subject: Re: Archive Any And All Text Usenet
Date: Sun, 10 Mar 2024 20:06:44 +0100
Organization: A noiseless patient Spider
Lines: 35
Message-ID: <usl0c4$34290$1@dont-email.me>
References: <FY2cnX6Uav6TOXH4nZ2dnZfqnPqdnZ2d@giganews.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 10 Mar 2024 19:06:44 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="845e6215760786f6cdaa9dc04d48a147";
	logging-data="3279136"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/0WQaOqwmyZYCD7f2b8JE+"
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:71dYSz4CDelihyCr7maLwKn1Yg8=
In-Reply-To: <FY2cnX6Uav6TOXH4nZ2dnZfqnPqdnZ2d@giganews.com>
Content-Language: en-US
Bytes: 2533

On 9/03/24 19:01, Ross Finlayson wrote:
> [snip]  I wonder how to define an
> "archive any and all text usenet", AAATU,
> filesystem convention, as a sort of "Library
> Filesystem Format", LFF.
> 
> The idea is that each "message", "post", has an ID,
> then as far as that's good, that each group
> in the hierarchy has a name, and that, each
> message has a date.  Then, the idea is to
> make an LFF, that makes a folder for a group,
> for a date, each its messages.
> 
> a.b.c/YYYY/MMDD/HHMM/

A filesystem is not a good match for all possible problems. Have you 
considered an SQL database, which IS a good match for a large number of 
problems?

> There are very useful notions of "mbox" and
> "maildir", with the idea that LFF or "maillff",
> and mbox and maildir variously have a great
> affinity.

These were a good idea when they were invented. SQL is a good idea now 
that it has also been invented. Most implementations do not suffer from 
the limitations you talk about below as well as other limitations you 
did not talk about below.

Another system you might be interested in is BitTorrent. I believe 
Library Genesis (an illegal backup of all published books) uses this for 
resilience. They divided the entire library into some number of torrents 
and then told people to go and seed the torrents so that if the library 
goes away, it can be reconstructed. Yours wouldn't be illegal, of course.