Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Rich <rich@example.invalid>
Newsgroups: comp.lang.tcl
Subject: Re: Thread with -async exits prematurely
Date: Wed, 19 Jun 2024 15:02:28 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 252
Message-ID: <v4uru4$21ajj$1@dont-email.me>
References: <6672b7d6$0$705$14726298@news.sunsite.dk>
Injection-Date: Wed, 19 Jun 2024 17:02:29 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="30a8cfe1c385c01859adef0a92bea9ae";
	logging-data="2140787"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX19mrXdGlZ0bTGI2/vmaUpRT"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.139 (x86_64))
Cancel-Lock: sha1:/PFJ2TW0xSsl0b+9rlK8cBWyOSQ=
Bytes: 10184

Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
> Hi all!
> 
> 
> My program is working fine when thread::send don't use the -async 
> option.  When it does, all of those created threads exit prematurely.
> 
> The pseudo-code I have is this:

Working code that you've tested to exhibit the bug you see is 
preferable, and your code was *very* close....

> ===== main file
> 
> while 1 {
>     ...
>     while {nr_live_threads < nr_max_threads} {

This will error out as a syntax error.  You want to both initialize 
these variables before you use them, and to interpolate them using $ 
above.

>         set tid [thread::create $init_script]
>         thread::send -async $tid [list sourceFiles ....]
>     }
>     after 10000
> }

You never increment nr_live_threads, so this loop above will (assuming 
the variables were initalized, and referenced, correctly) simply loop 
forever, creating new threads.  At least until the whole process is 
killed for using all free memory up.

> ===== oo.tcl
>     namespace eval ns0 {
>         proc runAnsible {...} {
>             Parse new ...
>             vwait ::exit_flag
>         }
>     }

You never signal to the master thread that this thread has exited, so 
the master (as written here) will never launch a new thread when an 
existing one finishes.

> This comprises the important parts of the script, I think.
> When thread::send does not use `-async`, the `vwait ::exit_flag` works and 
> the thread is run until the end.
> With `-async`, the thread exits shortly after the `thread::send` command.

Something must be different in the "psudeo" code vs. your real code 
then.

> I've read about `thread::preserve` and `thread::release`, but interpreted 
> it as necessary when threads have to be orchestrated and some may be 
> dependent on the results of others.

No, those are to do reference counting for thread cleanup.

> What I want is really to have several threads launched in the same moment, 
> at each run of the while loop that checks if the number of active threads 
> is less than the nr_max_threads.
> How can that be accomplished?

Well, first, you have to communicate the exit of a child thread back to 
he main thread, and have that comm path decrement "nr_active" (and you 
also need to increment nr_active when you launch a new thread).

Syntax cleaned up -- and simplified version of your original code, that 
*actually runs*:

thread-test:
    #!/usr/bin/tclsh

    package require Thread

    set nr_max_threads 4
    set nr_live_threads 0

    set init_script {
        puts stderr "Thread: [thread::id] Init: creating sourceFiles"
        proc sourceFiles {args} {
            source oo.tl
            ns0::runAnsible $args
        }
        puts stderr "Thread: [thread::id] waiting"
        thread::wait
        puts stderr "Thread: [thread::id] out of wait"
    }

    while 1 {
        while {$nr_live_threads < $nr_max_threads} {
            puts stderr "Main: live=$nr_live_threads max=$nr_max_threads"
            set tid [thread::create $init_script]
            incr nr_live_threads
            thread::send -async $tid [list sourceFiles ....]
        }
        puts stderr "Main: sleeping for 10s"
        after 10000
        puts stderr "Main: awake"
    }

oo.tl:
    namespace eval ns0 {
        proc runAnsible {...} {
            puts stderr "Thread [thread::id]: executing parse new"
            Parse new ...
            puts stderr "Thread [thread::id]: vwait begin"
            vwait ::exit_flag
            puts stderr "Thread [thread::id]: vwait complete"
        }
    }
    oo::class create Parse {
        constructor {...} {
            set random [expr {int(rand()*20000)}]
            puts stderr "Thread [thread::id]: object constructor - sleeping for $random"
            after $random [list set ::exit_flag 1]            
        }
    }

Sample run of the above:

    $ ./thread-test 
    Main: live=0 max=4
    Thread: tid0x7fbfa7b6c640 Init: creating sourceFiles
    Thread: tid0x7fbfa7b6c640 waiting
    Main: live=1 max=4
    Thread tid0x7fbfa7b6c640: executing parse new
    Thread tid0x7fbfa7b6c640: object constructor - sleeping for 8476
    Thread tid0x7fbfa7b6c640: vwait begin
    Thread: tid0x7fbfa6b6a640 Init: creating sourceFiles
    Thread: tid0x7fbfa6b6a640 waiting
    Main: live=2 max=4
    Thread tid0x7fbfa6b6a640: executing parse new
    Thread tid0x7fbfa6b6a640: object constructor - sleeping for 16806
    Thread tid0x7fbfa6b6a640: vwait begin
    Thread: tid0x7fbfa6369640 Init: creating sourceFiles
    Thread: tid0x7fbfa6369640 waiting
    Main: live=3 max=4
    Thread tid0x7fbfa6369640: executing parse new
    Thread tid0x7fbfa6369640: object constructor - sleeping for 11225
    Thread tid0x7fbfa6369640: vwait begin
    Thread: tid0x7fbfa5b68640 Init: creating sourceFiles
    Thread: tid0x7fbfa5b68640 waiting
    Main: sleeping for 10s
    Thread tid0x7fbfa5b68640: executing parse new
    Thread tid0x7fbfa5b68640: object constructor - sleeping for 5573
    Thread tid0x7fbfa5b68640: vwait begin
    Thread tid0x7fbfa5b68640: vwait complete
    Thread tid0x7fbfa7b6c640: vwait complete
    Main: awake
    Main: sleeping for 10s
    Thread tid0x7fbfa6369640: vwait complete
    Thread tid0x7fbfa6b6a640: vwait complete
    Main: awake
    Main: sleeping for 10s
    Main: awake
    Main: sleeping for 10s

And, it will continue to loop saying 'awake' and 'sleeping' since the 
exit of the children is never communicated to the master.

You need to master to become aware that one of the children has exited, 
so it knows to relaunch another child.  One way is to use the 
additional result variable for -async threads and vwait on that 
variable in the master.

Here is the 'diff' necessary to have the master monitor children 
exiting and to launch a new child when that happens:

    --- thread-test.v1	2024-06-19 10:42:34.359605931 -0400
    +++ thread-test	2024-06-19 11:00:01.433949725 -0400
    @@ -4,6 +4,7 @@
     
     set nr_max_threads 4
     set nr_live_threads 0
    +set sync 0
     
     set init_script {
         puts stderr "Thread: [thread::id] Init: creating sourceFiles"
    @@ -21,10 +22,10 @@
             puts stderr "Main: live=$nr_live_threads max=$nr_max_threads"
             set tid [thread::create $init_script]
             incr nr_live_threads
========== REMAINDER OF ARTICLE TRUNCATED ==========