Path: ...!local-4.nntp.ord.giganews.com!Xl.tags.giganews.com!local-3.nntp.ord.giganews.com!nntp.earthlink.com!news.earthlink.com.POSTED!not-for-mail
NNTP-Posting-Date: Tue, 19 Nov 2024 03:32:17 +0000
Subject: Re: The joy of pipes
Newsgroups: comp.os.linux.misc
References: <vgns2aqlhq@dont-email.me> <20241112111426.00007245@gmail.com>
 <e44df1bda1f1622a8d725c69860d3225@msgid.frell.theremailer.net>
 <m2ttc9y3d8.queerchen@cmschueller.my-fqdn.de> <eli$2411141855@qaz.wtf>
 <20241114160907.0000252b@gmail.com> <vh6a9k$33c17$5@dont-email.me>
 <hzSdnTUBKbG_YKv6nZ2dnZfqnPQAAAAA@earthlink.com> <A7GZO.66$hgYd.23@fx41.iad>
 <wwvr07bpizm.fsf@LkoBDZeT.terraraq.uk>
 <JJOdnfSeXoej5aT6nZ2dnZfqnPSdnZ2d@earthlink.com>
 <wwvwmh2z1y3.fsf@LkoBDZeT.terraraq.uk>
 <Uv-dnfY4yvgPJKf6nZ2dnZfqnPSdnZ2d@earthlink.com>
 <vhelgl$142m9$2@dont-email.me>
 <UricnZ0glICSeaf6nZ2dnZfqn_adnZ2d@earthlink.com>
 <vhf0sn$15fdk$1@dont-email.me>
From: "186282@ud0s4.net" <186283@ud0s4.net>
Organization: wokiesux
Date: Mon, 18 Nov 2024 22:32:17 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <vhf0sn$15fdk$1@dont-email.me>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Message-ID: <c1OdnZuXAslcm6H6nZ2dnZfqnPqdnZ2d@earthlink.com>
Lines: 64
X-Usenet-Provider: http://www.giganews.com
NNTP-Posting-Host: 99.101.150.97
X-Trace: sv3-yP934+6/KGMJevs9+Jj92q9WBTsVcsZ/nMWb4R1QrDzBgu2NOteEQT+CidFIesN+pdGCeDw8YOgXzqU!BBCf675tFhCu2+ok/PNYK/ZsEJJDIpjq/wmoIAVIC4tr0z9P0uU3CMkDpLvntJiU3n603wXoPPp8!qlSTB5PRLHWooFsAL4/E
X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
X-Postfilter: 1.3.40
Bytes: 4404

On 11/18/24 4:20 AM, Pancho wrote:
> On 11/18/24 06:51, 186282@ud0s4.net wrote:
>> On 11/18/24 1:06 AM, Lawrence D'Oliveiro wrote:
>>> On Sun, 17 Nov 2024 22:50:41 -0500, 186282@ud0s4.net wrote:
>>>
>>>> As I said somewhere, if your child processes are sending megabytes
>>>> back to the parent you're DOING SOMETHING WRONG.
>>>
>>> Says the one who has no clue how to write real-world programs.
>>
>>    Am I supposed to say something horrible about
>>    you now ?
>>
>>    Nope. Won't. I'm sick of 'The Wars'.
>>
>>    Clue - I've writ 'real world programs' and
>>    got paid well for them from the punch-card
>>    days on. I did it my way, for my reasons,
>>    my 'vision'.
>>
>>    And so did you.
> 
> I mainly did as I was told. In large political organisations following 
> your own vision can be problematic. You have to fight for a vision.


   I very intentionally stayed away from such orgs. A little
   less money, but the happiness factor more than compensated.
   Got to pursue many esoteric projects - some of which WERE
   useful - and the org paid for most of it (except the odd
   Radio Shack parts).


> I've no idea why using IPC to send megabytes of data between different 
> processes is wrong. That is what many service-orientated architectures 
> do. The moment you have a persistence service it is likely to happen.

   Really you DON'T want to send megabytes as IPC ... but
   somebody somewhere MAY wanna. So, what's the best way
   for them to do that ? Don't necessarily piss on 'em but
   help 'em out instead.

> Although to be fair we very rarely used pipes, directly, almost never. 
> It was always something like REST or message queues.

   Again though, just "blocks of information". MANY ways
   to do that. Each developer has to figure out what's
   best for THEIR particular app.

   I mentioned a server style app I made, but I wanted the
   child processes/threads (tried both ways) to return
   some active stat data to the parent. Not a huge amount
   of info, but useful. Pipes were the best way. The
   parent was allowed to xmit a 'kill' to a child in
   trouble, via the pipe, so it could shut down in an
   orderly fashion.

   Pre-threaded servers seem to have the highest capacity
   and throughput. The parent allocates maybe ten threads
   to start off with and then can add new blocks of ten
   as-needed. It's a little harder to manage than some
   other kinds of servers, and sometimes a bit harder
   to visualize how it's working, but they're good things.
   Once you've got it you're SET.