Path: ...!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Pancho <Pancho.Jones@proton.me>
Newsgroups: comp.os.linux.misc
Subject: Re: The joy of pipes
Date: Mon, 18 Nov 2024 11:05:03 +0000
Organization: A noiseless patient Spider
Lines: 38
Message-ID: <vhf70v$16jpu$1@dont-email.me>
References: <vgns2aqlhq@dont-email.me> <20241112111426.00007245@gmail.com>
 <e44df1bda1f1622a8d725c69860d3225@msgid.frell.theremailer.net>
 <m2ttc9y3d8.queerchen@cmschueller.my-fqdn.de> <eli$2411141855@qaz.wtf>
 <20241114160907.0000252b@gmail.com> <vh6a9k$33c17$5@dont-email.me>
 <hzSdnTUBKbG_YKv6nZ2dnZfqnPQAAAAA@earthlink.com> <A7GZO.66$hgYd.23@fx41.iad>
 <wwvr07bpizm.fsf@LkoBDZeT.terraraq.uk>
 <JJOdnfSeXoej5aT6nZ2dnZfqnPSdnZ2d@earthlink.com>
 <wwvwmh2z1y3.fsf@LkoBDZeT.terraraq.uk>
 <Uv-dnfY4yvgPJKf6nZ2dnZfqnPSdnZ2d@earthlink.com>
 <vhelgl$142m9$2@dont-email.me>
 <UricnZ0glICSeaf6nZ2dnZfqn_adnZ2d@earthlink.com>
 <vhf0sn$15fdk$1@dont-email.me> <vhf1qg$165g7$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Mon, 18 Nov 2024 12:05:03 +0100 (CET)
Injection-Info: dont-email.me; posting-host="4bb256e55187fc1c7fdad52720ab946c";
	logging-data="1265470"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/6SAMIa7BzdvLlEZjyXPZhNNvHaf+2DM8="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:yiJJUk0KBqSo+LXzmZQGE8KcWRg=
Content-Language: en-GB
In-Reply-To: <vhf1qg$165g7$1@dont-email.me>
Bytes: 3397

On 11/18/24 09:36, Lawrence D'Oliveiro wrote:
> On Mon, 18 Nov 2024 09:20:23 +0000, Pancho wrote:
> 
>> I've no idea why using IPC to send megabytes of data between different
>> processes is wrong.
> 
> It is something I have done. It’s perfectly commonplace.
> 
>> Although to be fair we very rarely used pipes, directly, almost never.
> 
> I have used pipes, I have used Unix sockets, I have used network sockets.
> 
> If you are running a Linux GUI, then almost certainly it is built on D-Bus
> as a high-level IPC mechanism that is used as a core component. That is
> designed to run over Unix sockets. It is not itself designed for high-
> bandwidth data transfers; if you want to do that, you can exchange your
> own D-Bus messages to set up custom pipe or Unix socket connections
> between bus peers.
> 
>> It was always something like REST or message queues.
> 
> Message queues are an OS-provided primitive, but REST is not -- that is a
> protocol, not a transport. What transport(s) did you use for that? I would
> assume network connections.

I don't know if Linux provides message queues as a primitive, or not. I 
meant I used message queue protocols (MQTT,zeroMQ, ActiveMQ).

I guess I was saying as a software developer I only cared about the 
application layer, I never went down to the level of using pipes 
directly. I would have thought my experience was reasonably normal, for 
the last few decades. The period after software developers needed to 
roll everything themselves.

I do remember looking at named pipes and deciding it was appropriate for 
some task, but I have no memory of actually implementing it. Which could 
mean it just worked and I never thought about it again, or it could mean 
I didn't do it.