Path: ...!2.eu.feeder.erje.net!feeder.erje.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: BGB <cr88192@gmail.com>
Newsgroups: comp.arch
Subject: Re: Article on new mainframe use
Date: Fri, 23 Aug 2024 02:31:46 -0500
Organization: A noiseless patient Spider
Lines: 218
Message-ID: <va9dt6$r72m$1@dont-email.me>
References: <v9iqko$h7vd$1@dont-email.me>
 <bb873f7f6a14f222f73abacd698e60eb@www.novabbs.org>
 <3f8sbj9chugcr6arbpck2t7nb0g87ff6ik@4ax.com>
 <f7fe11f84f9342f0a7e27d4a729aadad@www.novabbs.org>
 <li71t8Fs9jnU1@mid.individual.net> <v9mc57$15mm9$2@dont-email.me>
 <kmvubjdn7ub4bkgfhpj89c5vsl37vpp16d@4ax.com> <va88rq$ioap$1@dont-email.me>
 <va8sji$otgt$5@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 23 Aug 2024 09:31:51 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="44f150c550c51d8a070b78e2472a6035";
	logging-data="891990"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX19X2l7XKlAs4HiNqbH1PsXbS4+KoVAAh4w="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:ph1rsJwifFX9EP7IT/qnNPTfVvE=
Content-Language: en-US
In-Reply-To: <va8sji$otgt$5@dont-email.me>
Bytes: 8826

On 8/22/2024 9:36 PM, Lawrence D'Oliveiro wrote:
> On Thu, 22 Aug 2024 15:59:34 -0500, BGB wrote:
> 
>> Underused: It is a sensible way of structuring data, but needing to
>> interface with it though SQL strings is awkward and inefficient.
> 
> Actually, that works fine in any language with decent string handling.
> 

String processing adds bulk and inefficiency.
Granted, maybe not enough to matter in the face of a typical database query.

But, I was left thinking, some programs use SQLite, which exists as a 
single giant C file. I guess it technically works, but has the downside 
of adding something like IIRC around 900K or so to the size of the 
binaries' ".text" section.


In my recent projects, I am left being a little more aware of how much 
code footprint and space everything costs (along with the limitations of 
trying to keep things performing acceptably at 50 MHz).



But, say, a goal would be to try to fit a semi-usable database engine 
into say a few kLOC or so (rather than several hundred kLOC).

Things like text parsing and other complex high level features would be 
on a high priority list of things to jettison.


Some can maybe still be present in the code, but:
If they are API calls, then if they are never called, code reachability 
can cull them;
If there is a text parser, and they are reachable from said parser, and 
the program uses said parser, one is SOL, code reachability will need to 
include the whole thing...

I guess, similar reasons for why one doesn't want to add too many 
complex string formatting features to "printf()" (since, if they are not 
used, they uselessly add bulk to any program that uses "printf()" and 
similar).



Similar related reasons for why TKGDI's built in image loading doesn't 
support JPG or PNG. Granted, OTOH, QOI is a bit niche, but decodes 
quickly and doesn't need all that much code (vs a PNG decoder).

Similar reasons for why I am still going with BMP for stuff, and am 
making use of CRAM (including in non-standard CRAM BMP images). Like, as 
much as CRAM "kinda sucks", it is fast and needs relatively little code 
for a decoder.


I still lack a good "cheap" JPEG stand-in. I had defined LCIF, which was 
essentially "QOI but with some color-cell stuff glued on". OK, but still 
not an ideal JPEG stand-in (one might want something that can be 
lossless or near lossless; LCIF's "top end" quality still basically 
looks like DXT1).

Then LCIF was left in competition with the "cheaper" option of CRAM+RP2 
(where, if I already have a CRAM decoder and an RP2 decoder, RP2 encoded 
CRAM in a BMP doesn't add too much additional code footprint). Though, 
both top-end quality and Q/bpp are worse with CRAM.

Basic CRAM is ~ 2.0bpp, but with RP2 compression may be ~ 0.25 to 2.0 
bpp depending on how LZ compressible the image is (and RP2 tends to beat 
LZ4 in this case). Works OK for UI graphics.



I have some ideas I could try, if I wanted to do something vaguely JPEG 
like but at a lower line count. But, in any case, if it goes too much 
over ~ 1k lines, it becomes moot (at around 2k lines, that is the size 
of a JPEG decoder).

Though, minimizing line-count would likely mean the "semi-crappy" option 
of probably using Rice coding and a block Haar transform.

Using 12-bit length limited Huffman would give better compression, but 
(unlike Rice) would still need around 250 lines for the entropy coder 
(vs ~ 100 or so for Rice, mostly due to bitstream code, like 
PeekBits/SkipBits/ReadBits functions).


Well, and my kernel has already gotten fairly large, due to the existing 
overhead needed for GUI and GL backend stuff (it becomes tempting to try 
to fold this off into a DLL).


>> Or, do some weird OO thing where object methods exist for SQL keywords
>> and each method returns an object that can be used to glue on the next
>> keyword:
>>     res=db.select().from("FOOTAB").where().equals("NAME", "John").end();
> 
> ORMs are a waste of time.

Yeah, this is not an API design of honor...

Granted, trying to awkwardly map it to an OpenGL style API design 
wouldn't be much better, say:
   dbmsBeginSelect();
   dbmsFromTable("FOOTAB");
   dbmsWhere(DBMS_EQUAL, "NAME", "John");
   dbmsEndSelect();
   while(dbmsPollResult())
   {
     dbmsGetResultStr("NAME", dbFirstName);
     dbmsGetResultStr("LAST", dbLastName);
     ...
     dbmsNextResult();
   }


One other possibility might be something like, say:
   DBMS_HTABLE hFootab;
   DBMS_HKEYELEM aSelKey[8];
   DBMS_HVALELEM aSelVal[8];
   DBMS_HENUM    aSelOpr[8];
   DBMS_HKEYELEM aResKey[12];
   DBMS_HVALELEM aResVal[12];
   FBMS_HQUERY   hRes;
   int           nResCol;
   int i, j, k;

   aSelKey[0]=dbmsGetHKey(db, "NAME");
   aSelVal[0]=dbmsGetHString(db, "John");
   aSelOpr[0]=DBMS_EQUAL;

   hFooTab=dbmsOpenTable(db, "FOOTAB");
   hRes=dbmsTableSelect(db,
     hFooTab, aSelKey, aSelOpr, aSelVal, 1, NULL, 0);

   nResCol=12;  //max of 12 columns
   dbmsGetResultRowKeys(db, hRes, aResKey, &nResCol);
   for(i=0; i<nResCol; i++)
   {
     str=dbGetKeyName(db, aResKey[i]);
     printf("%s  ". str);
   }
   printf("\n");
   while(dbmsPollResultNotDone(db, hRes))
   {
     dbmsReadResults(db, hRes, aResKey, aResVal, nResCol);
     // do something with results
     for(i=0; i<nResCol; i++)
     {
       str=dbGetValueAsStringTemp(db, aResVal[i]);  //*1
       printf("%s  ". str);
     }
     printf("\n");
   }
   dbmsCloseResults(db, hRes);

*1: Where the string pointer will be valid until either the next results 
are read or the results are closed. Similarly, any values read from the 
results will only be valid until either the next results are read or the 
results are closed.


Such an API design would still be awkward, but could potentially be 
lower overhead than some other options.



Skimming SQLite DB format:
Why such large nodes?...
I would have thought maybe 4K, not 32K or 64K...
Then again, maybe bigger might be better for less row-related fragmentation.
Big Endian for some reason.


========== REMAINDER OF ARTICLE TRUNCATED ==========