Path: ...!2.eu.feeder.erje.net!feeder.erje.net!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: vallor <vallor@cultnix.org>
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: 23 Nov 2024 03:25:08 GMT
Lines: 78
Message-ID: <lqd08jFisv3U1@mid.individual.net>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
	<vhf5ts$16rpr$1@dont-email.me> <vhfjtj$19ijm$1@dont-email.me>
	<vhja6j$23f5e$4@dont-email.me> <vhmm4c$hnbj$1@dont-email.me>
	<vhmn2t$hv8i$3@dont-email.me> <vhnikj$me7m$1@dont-email.me>
	<wwvcyio8m9p.fsf@LkoBDZeT.terraraq.uk> <vhorrb$t48o$1@dont-email.me>
	<vhosra$1171f$1@dont-email.me> <lqalg1F7fi9U2@mid.individual.net>
	<vhql0r$1a0ch$2@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Trace: individual.net rEeaVJheTwwExzIAMeR8PwjYiylYVUQnuzBxNk+SIC4sWFMFHm
Cancel-Lock: sha1:S2z6t591ODHaFuuosqxgI2Pf9Nw= sha256:rNtion57xfWI2cfl2HxYpjqoUyjytqZPXOEW0ZwHquE=
X-Face: +McU)#<-H?9lTb(Th!zR`EpVrp<0)1p5CmPu.kOscy8LRp_\u`:tW;dxPo./(fCl
 CaKku`)]}.V/"6rISCIDP`
User-Agent: Pan/0.161 (Hmm2; be402cc9; Linux-6.12.0)
Bytes: 4335

On Fri, 22 Nov 2024 19:11:23 -0000 (UTC), Rich <rich@example.invalid>
wrote in <vhql0r$1a0ch$2@dont-email.me>:

> vallor <vallor@cultnix.org> wrote:
>> On Fri, 22 Nov 2024 03:12:43 -0000 (UTC), Lawrence D'Oliveiro
>> <ldo@nz.invalid> wrote in <vhosra$1171f$1@dont-email.me>:
>> 
>>> On Thu, 21 Nov 2024 21:55:37 -0500, Phillip Frabott wrote:
>>> 
>>>> We had to drop named pipes solely because of the performance hit
>>>> because it is writing to a file system so it's being controlled by
>>>> the file system, even if that file system is in memory.
>>> 
>>> That doesn’t make any sense, if we were talking about Linux. Is this
>>> on Windows, by any chance?
>> 
>> Doesn't the named pipe connection work through the filesystem code?
>> That could add overhead.
> 
> Only to the extent that a filesystem lookup has to occur to lookup the
> name in order to open() the name.
> 
> Once you have a file descriptor back from the open() call, there is no
> difference at all kernel wise betwenn the two, they are one and the same
> block of kernel code.

I stand corrected about that.

> 
>> Can't use named pipes on just any filesystem -- won't work on NFS for
>> example, unless I'm mistaken.
> 
> Correct, you need a filesystem that supports storing a 'name' that that
> is a reference to a pipe, so windows filesystems are out.
> 
> Named pipes appear as 'pipe' nodes across NFS (just tested this to be
> certian).  And, so long as all the "accessors" of the named pipe are
> running on the same Linux machine with the NFS mount containing the pipe
> node, the named pipe works as expected (just tested this as well).

I tested it too (with an NFS v4.1 filesystem), and yes, mkfifo makes
a named pipe, and it works as expected.  (Didn't expect it to work
across machines, though that would be a neat trick.)


> But a named pipe on NFS does not give you a machine to machine (two
> different machines) transmit channel.
> 
>>>> As the demand grows, we are actually at the limits of performance
>>>> that even unnamed pipes gives us. So we are starting to migrate to
>>>> UNIX sockets which has about double to bandwidth and performance of
>>>> pipes.
>>> 
>>> Not sure how that works, given that Unix sockets are actually a more
>>> complex mechanism than pipes.
>> 
>> With Unix sockets, once the connection is made, it's all in-memory
>> networking.
> 
> Correct.
> 
>> I suspect (but don't know) that named pipes require the data to pass
>> through the filesystem for each write.
> 
> Incorrect.  The only 'filesystem' access for named pipes is during the
> open() call to look up the name from the filesystem.  Once you get the
> file descriptor back, it is the exact same in-memory FIFO queue as an
> anonymous pipe created via pipe() (at least on Linux).

Again, I stand corrected on that.

(Haven't figured out how to increase ulimit -p yet, doesn't seem
to want to increase, even as root...)

-- 
-v System76 Thelio Mega v1.1 x86_64 NVIDIA RTX 3090 Ti
   OS: Linux 6.12.0 Release: Mint 21.3 Mem: 258G
   "A good hot dog feeds the hand that bites it."