Path: ...!news.mixmin.net!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail From: Rainer Weikusat Newsgroups: comp.unix.programmer Subject: Re: Tale of a Fork Date: Fri, 17 Jan 2025 22:40:54 +0000 Lines: 57 Message-ID: <87a5bpkqrt.fsf@doppelsaurus.mobileactivedefense.com> References: <87plky7auh.fsf@doppelsaurus.mobileactivedefense.com> Mime-Version: 1.0 Content-Type: text/plain X-Trace: individual.net jH25Lj5SlLUKGubyhLxY2AAIm3eWzGYliZTd/Ixyjzlix7oaM= Cancel-Lock: sha1:Ur1MfpFC6ljSfQUMIlkuBfIrsQA= sha1:1HvHyrX/rjIlttTHR87H29fYHdM= sha256:o4WW/vGLElxKsy2gKzrQECE2nIgZRtluzN2T0AKqFGw= User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Bytes: 3812 Rainer Weikusat writes: > The purpose of this post (as with its predecessors although it has a > while) is to collect use-cases for fork which don't involve invoking > exec in the forked process to run another program. [...] > 3. Fork to have the old program running in a new process where it can > continue to receive and buffer network input and will also keep the > running state for later restoration. This deserves a follow-up: Another feature this program also supports is file uploads. As ActionCable does unlimited buffering (read: buffers a lot, exact limit unknown) and it's supposed to be possible to cancel uploads, flow control is necessary here so that sender and receiver operate at least roughly in real-time wrt each other. This uses a fixed-window-based algorithm where the sender sends data blocks until it has completely filled the window and each data block is acknowledged by the receiver after it was actually written to the file. Each ack received by the sender increases the available window by one block. ActionCable is essentially a chat protocol following a so-called pub/sub model. Clients can subscribe to so-called channels and will then receive everything published to such a channel after they have subscribed to it. This implies that something must remain subscribed to the channel and receive upload data messages while the updated program is still busy with initializing itself. Initialization requires 1. Starting perl. 2. Loading and compiling a lot of perl code, roughly 10.000 LOC in total. 3. Resolve the name of the cloud endpoint via DNS. 4. Establish a TCP connection it. 5. Negotiate TLS. 6. Do a WebSocket handshake to switch to WebSocket. 7. Receive an ActionCable greeting message. 8. Do a handshake in order to subscribe to the channel. 9. For each file upload, do another handshake to subscribe to the file upload channel for this upload. 9 is necessary because ActionCable is essentially a virtual, dumb repeater, ie, it sends all messages received for a channel to all subscribers, including to the party which originally sent the message. As this takes (for computers) a considerable amount of time, it's vital that something remains subscribed to the channel in order to receive file data messages the sender sent because it still has window space available. That's done by the original program running in the forked process. In total, this arrangement works like charm: Running file uploads just continue despite the original program running in the original process was meanwhile completely replaced. The fork/exec split may have been invented accidentally (or rather, was invented accidenally) because of path-of-least-resistance programming in an early UNIX version but it's decidedly a genuine discovery: Two independent system primitives which can be combined into a whole which is more than just it parts. No wonder that people hate it so much.