Path: ...!2.eu.feeder.erje.net!feeder.erje.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: anton@mips.complang.tuwien.ac.at (Anton Ertl)
Newsgroups: comp.arch
Subject: Re: In-memory database (was: big, fast, etc, was is Vax addressing sane today)
Date: Sat, 14 Sep 2024 10:45:34 GMT
Organization: Institut fuer Computersprachen, Technische Universitaet Wien
Lines: 51
Message-ID: <2024Sep14.124534@mips.complang.tuwien.ac.at>
References: <vbd6b9$g147$1@dont-email.me> <vbtqib$2sce$2@dont-email.me> <vbvhs3$2std$1@gal.iecc.com> <20240913122217.00002a21@yahoo.com> <vc2c2n$2prb$1@gal.iecc.com> <2024Sep14.112146@mips.complang.tuwien.ac.at> <vc3mqu$1df1t$1@dont-email.me>
Injection-Date: Sat, 14 Sep 2024 13:39:58 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="6db5ee5180e7e79132e55bada9f3289b";
	logging-data="1533181"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX19RUwFJpchb3PNQ9z2vu2gv"
Cancel-Lock: sha1:RJAXXZH8zOLZ90cy+UBSEwQnNKA=
X-newsreader: xrn 10.11
Bytes: 3528

Thomas Koenig <tkoenig@netcologne.de> writes:
>Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>
>[in-memory database]
>
>> but the question is if
>> the machine has enough RAM for the database.  Our dual-Xeon system
>> from IIRC 2007 has 24GB of RAM, not sure how big it could be
>> configured; OTOH, we have a single-Xeon system from 2009 or so with
>> 32GB of RAM (and there were bigger Xeons in the market at the time).
>
>The minimum requirement of SAP HANA is 64 GB of memory, but typical
>ranges are from 256GB to 1TB.

What is the relevance of SAP HANA for the topic at hand?

The question was if the RAM can hold the data.  For each account they
would have to keep the current balance (64 bits should be enough for
that), the account number (64 bits for the up to 19 digits of a Visa
card) for verifying that we are at the correct entry in the hash table
and probably some account status information (64 bits should be
plenty?).

There is also the sequence of transactions (a 64-bit transaction
offset in the log per transaction should be enough for that).  The
sequence of transactions may be useful for fraud detection, but I
don't know enough about that to know how to scale the system, so I'll
just say that fraud detection is done by a bigger system before the
transaction goes through to the transaction processing computer.

The sequence of transactions is also needed for generating the reports
and for dealing with customer complaints, but again, that's not
processing the transactions themselves (and is basically read-only,
except that the customer-complaint processing may result in additional
transactions).

So, with 24 bytes needed for each account on the
transaction-processing server, 32GB with, say 8GB left for
copy-on-write and other administrative purposes should be good for
about 900M accounts at a hash table load factor of 84%.  I guess that
Visa has more accounts, so one would need a box with more RAM.

A single core of the Xeon should easily be able to handle all the 56K
transactions per second, both the logging and the update of the hash
table, and in that case no locking is needed.  But that first needs a
sequence of transactions coming in.

- anton
-- 
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
  Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>