Path: ...!feeds.phibee-telecom.net!3.eu.feeder.erje.net!feeder.erje.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: David Brown <david.brown@hesbynett.no>
Newsgroups: comp.lang.c
Subject: Re: ASCII to ASCII compression.
Date: Fri, 7 Jun 2024 16:45:03 +0200
Organization: A noiseless patient Spider
Lines: 64
Message-ID: <v3v6df$23q0b$1@dont-email.me>
References: <v3snu1$1io29$2@dont-email.me> <v3spmv$1jbjq$1@dont-email.me>
 <v3t150$1kia9$1@dont-email.me> <v3ukbb$20s0s$3@dont-email.me>
 <v3uva6$22nnp$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 07 Jun 2024 16:45:04 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="ec5c1f9082215424827ae25d429fe7a7";
	logging-data="2222091"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18IDnZrFl7vc6+Waf6HDROtb+En+UfPuSY="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Cancel-Lock: sha1:rstGqkwI44wKOE8C7tLucj2HsBM=
Content-Language: en-GB
In-Reply-To: <v3uva6$22nnp$1@dont-email.me>
Bytes: 3980

On 07/06/2024 14:43, Malcolm McLean wrote:
> On 07/06/2024 10:36, David Brown wrote:
>> On 06/06/2024 21:02, Malcolm McLean wrote:
>>> On 06/06/2024 17:55, bart wrote:
>>>> On 06/06/2024 17:25, Malcolm McLean wrote:
>>>>>
>>>>> Not strictly a C programming question, but smart people will see 
>>>>> the relavance to the topicality, which is portability.
>>>>>
>>>>> Is there a compresiion algorthim which converts human language 
>>>>> ASCII text to compressed ASCII, preferably only "isgraph" characters?
>>>>>
>>>>> So "Mary had a little lamb, its fleece was white as snow".
>>>>>
>>>>> Would become
>>>>>
>>>>> QWE£$543GtT£$"||x|VVBB?
>>>>
>>>> What's the problem with compressing to binary (using existing, 
>>>> efficient utilities), then turning that binary into ASCII (like Mime 
>>>> or Base64)?
>>>>
>>> Because if a single bit flips in a zip archive, it's likely the 
>>> entire archive will be lost. This scheme is robust. We can emed 
>>> compressed text in programs, and if it is corruped, only a single 
>>> line will become unreadable.
>>
>> Ah, you want something that will work like your newsreader program 
>> that randomly changes letters or otherwise corrupts your spelling 
>> while leaving most of it readable?  :-)
>>
>> Pass the data through a compressor and then add forward error checking 
>> mechanisms such as Reed-Solomon codes.  Then convert to ASCII base64 
>> or similar.
>>
> Yes, exactly.
> 
> I want a system for compression which is robust to corruption, can be 
> stored as text, and with a compressor / decompressor which can be 
> written by a child hobby programmer with only a very little bit of 
> experience of programming.
> 

That last "requirement" is completely unrealistic.  Forget it.  Then you 
already have a solution, as I outlined above.

I don't think it is remotely helpful to have error correction in your 
format.  You have to handle email extraordinarily badly to have any 
issues transferring 8-bit binary data, and you certainly won't have 
trouble if you are using Base64.  Either the email will arrive 
correctly, or it will not arrive at all.  At most, you could add a CRC 
check after compressing and before Base64 encoding.

But then, I don't think any of this stuff will be remotely useful in 
practice.  But if you are enjoying working on it, that's all the 
motivation and justification anyone could ever need.  So if you want 
error correction and compression, that's fine.

> That's what I need for Baby X. The FileSystem XML files can get very 
> large, and of course Baby X programmers are going to ask about 
> compression. And I don't think there is an existing system, and so I 
> shall devise one.
>