Path: ...!news.misty.com!weretis.net!feeder8.news.weretis.net!usenet.goja.nl.eu.org!aioe.org!UvG48Vl/yBcPk1+31nlajw.user.46.165.242.75.POSTED!not-for-mail
From: Helmut Waitzmann <nn.throttle@xoxy.net>
Newsgroups: comp.unix.shell
Subject: Redirecting file descriptors (was: How to redirect to non-standard file descriptors?)
Date: Tue, 05 Oct 2021 21:55:27 +0200
Organization: Aioe.org NNTP Server
Message-ID: <835yubi6vk.fsf@helmutwaitzmann.news.arcor.de>
References: <irtteiFpfsqU1@mid.individual.net>
Reply-To: Helmut Waitzmann Anti-Spam-Ticket.b.qc3c <oe.throttle@xoxy.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Injection-Info: gioia.aioe.org; logging-data="55094"; posting-host="UvG48Vl/yBcPk1+31nlajw.user.gioia.aioe.org"; mail-complaints-to="abuse@aioe.org";
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (gnu/linux)
Mail-Reply-To: Helmut Waitzmann Anti-Spam-Ticket.b.qc3c <oe.throttle@xoxy.net>
Mail-Copies-To: nobody
Cancel-Lock: sha1:HfW0lRP87AGuti5GVN7YTNYaW6w=
X-Notice: Filtered by postfilter v. 0.9.2
Bytes: 6772
Lines: 131

Robert Latest <boblatest@yahoo.com>:

>I must say I understand very little of the redirection chapter in=20
>"man bash".

That's because the bash manual page purpose is not to teach the unix=20
file descriptor redirection mechanism.=C2=A0 The bash manual page assumes=20
that the reader already is familiar with the unix file descriptors=20
and knows what =E2=80=9Credirection=E2=80=9D of those file descriptors is.

So, here is a very short introduction to unix file descriptors:=20


For accessing files, the kernel maintains two data structures: a=20
system-wide table (or array) of opened files, and a per-process=20
(i.=E2=80=AFe.=C2=A0each process has one of its own) table (or array) of fi=
le=20
descriptors.=C2=A0 When a process asks the kernel to open a file, for=20
example by invoking the =E2=80=9Copen()=E2=80=9D system service, the kernel=
 picks an=20
unused entry of the process' table of file descriptors.=C2=A0 This entry=20
will be identified by its position (or index) in that file=20
descriptor table: a small number (0, 1, 2, =E2=80=A6), a.=E2=80=AFk.=E2=80=
=AFa.=C2=A0the file=20
descriptor number.

Then the kernel picks an unused entry of the system-wide table of=20
open files and records a reference to (for example: the position in=20
the system=E2=80=90wide open=E2=80=90files table) that entry in the allocat=
ed entry=20
of the process' table of file descriptors.

In the allocated entry of the system-wide table of open files, the=20
kernel records, which file is to be accessed, the access mode (that=20
is, whether the file is opened for reading or for writing, if=20
writing will overwrite the file or append to it, etc.), the current=20
access position in the opened file, and, how many file descriptor=20
table entries are associated with this entry of the system-wide=20
table of open files (in this case: 1).

Finally, the kernel returns the index of the allocated entry of the=20
process' file descriptor table to the process.=C2=A0 In the bash manual=20
page that index is known as file descriptor number.

For example, the return value of the system call =E2=80=9Copen()=E2=80=9D w=
ill=20
return such a number.=C2=A0 See the manual page =E2=80=9Copen(2)=E2=80=9D.

Note:=C2=A0 The process can't (directly) tell the kernel, which entry=20
of the file descriptors table to use, when opening a file.

But there is a system service, which allows a process to tell the=20
kernel, which entry of the file descriptors table to use: =E2=80=9Cdup2()=
=E2=80=9D.=C2=A0=20
See the manual page =E2=80=9Cdup2(2)=E2=80=9D.=C2=A0 The =E2=80=9Cdup()=E2=
=80=9D and =E2=80=9Cdup2()=E2=80=9D system=20
services essentially copy one entry of the process' file descriptor=20
table to another.


So, if you tell the shell to start a command by=20


   command > output.txt

the shell will first open the file =E2=80=9Coutput.txt=E2=80=9D by means of=
 the=20
system service =E2=80=9Copen=E2=80=9D, which will (for example) return the =
file=20
descriptor number 42.=C2=A0 Then the shell will tell the kernel by means=20
of the system service =E2=80=9Cdup2=E2=80=9D to copy the contents of the en=
try #42=20
of the process' file descriptor table to the entry #1 in the same=20
table.=C2=A0 Finally the shell will tell the kernel by means of the=20
system service =E2=80=9Cclose=E2=80=9D to release (i.=E2=80=AFe.=C2=A0empty=
) the entry #42.

The result of those three system service calls is, that the file=20
descriptor #1 will refer to the opened file =E2=80=9Coutput.txt=E2=80=9D (r=
ather=20
than to the terminal):=C2=A0 When the command writes data to its file=20
descriptor #1 (a.=E2=80=AFk.=E2=80=AFa.=C2=A0standard output), the data wil=
l arrive in=20
the file =E2=80=9Coutput.txt=E2=80=9D.=C2=A0 That's why this sequence of sy=
stem service=20
calls is often called redirection.=C2=A0 But note:=C2=A0 There is nothing l=
ike=20
redirection or forwarding involved in this mechanism.=C2=A0 It's just the=20
effect of copying one file descriptor table entry to another.

Now, if the command is=20


>    command > output.txt 2>&1
>

the shell first will do the same and then tell the kernel by means=20
of the system service =E2=80=9Cdup2=E2=80=9D to copy the contents of the en=
try #1 to=20
the entry #2.=C2=A0 Now two entries in the process' file descriptor=20
table=C2=A0=E2=80=93 #1 and #2=C2=A0=E2=80=93 refer to the opened file =E2=
=80=9Coutput.txt=E2=80=9D.

On the other hand, look, what=20


>    command 2>&1 > output.txt
>

would do:=C2=A0 It would copy the entry #1 of the process' file=20
descriptor table to the entry #2 and then open the file=20
=E2=80=9Coutput.txt=E2=80=9D, thus getting the (hypothetical) entry #42, th=
en copy=20
that entry #42 to the entry #1.=C2=A0 Of course the entry #2 won't be a=20
copy of the entry #42 then, i.=E2=80=AFe.=C2=A0won't refer to the file=20
=E2=80=9Coutput.txt=E2=80=9D.

>It doesn't seem logical that a redirection specifier after the=20
>target file would influence what goes into that file.

You are fooled by the misnomer file descriptor redirection.=C2=A0 Keep in=20
mind, that

   2>&1

is essentially not much more than copying entry #1 to entry #2.=C2=A0=20
There is no forwarding involved like =E2=80=9Cif you want to write to the=20
file referred by the process' file descriptor table entry #2, look=20
up the file descriptor table entry #1 and use that instead=E2=80=9D.

See also <https://en.wikipedia.org/wiki/File_descriptor#top>.=20