Path: ...!goblin2!goblin.stu.neva.ru!aioe.org!eternal-september.org!feeder.eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: Ruvim <ruvim.pinka@gmail.com>
Newsgroups: comp.lang.forth
Subject: Re: Recognizers - an alternative
Date: Sat, 24 Oct 2020 17:14:31 +0300
Organization: A noiseless patient Spider
Lines: 160
Message-ID: <rn1csa$b02$1@dont-email.me>
References: <rm4260$3fu$1@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 24 Oct 2020 14:14:34 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="8532595ce0a0396ed06af54b584838df";
	logging-data="11266"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX19zsYZ0g1yi92kyR++j9vnb"
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1
Cancel-Lock: sha1:Bs+N7zfp4ZpMJY5se89cc0BBHfM=
In-Reply-To: <rm4260$3fu$1@dont-email.me>
Content-Language: en-US
X-Mozilla-News-Host: news://nntp.aioe.org
Bytes: 7155

On 2020-10-13 14:14, Gerry Jackson wrote:
> There have been four recognizer proposals, one an RfD and three prior to 
> the September Forth 200X meeting. These are:
> 
> 1. the official RfD 
> http://amforth.sourceforge.net/pr/Recognizer-rfc-D.html (this link 
> doesn't seem to be working at present, I don't know of another)
> 2. Andrew Haley's simple recognizers 
> https://forth-standard.org/proposals/an-alternative-to-the-recognizer-proposal#reply-493 
> 
> 3. Anton Ertl's Nestable recognizer sequences 
> https://forth-standard.org/proposals/nestable-recognizer-sequences#contribution-149 
> 
> 4. Bernd Paysan's Minimalistic API 
> https://forth-standard.org/proposals/minimalistic-core-api-for-recognizers#reply-515 

[...]

> Of the proposals it seems to me that Andrew's  approach is better 
> because of its simplicity, ease of implementation and the flexibility 
> that it gives users when defining recognizers.
> 
> As an experiment to see how well it works I've implemented a scheme 
> similar to Andrew's.

I would prefer to also see the code :) Since your description leaves 
some parts too unclear. Gist at GitHub is a useful platform for that.
https://gist.github.com/


[...]
> I didn't find it necessary to pass a mode value to the recognizer as 
> Andrew proposed but one could easily be added.

It's that since in the API of Andrew a user provides not a recognizer 
for lexeme, but a translator for lexeme.  You don't need to know the 
state for recognizing. But you need to know the state for translation 
(i.e. to perform interpretation or compilation semantics depending on 
STATE).

In general, the state can be passed explicitly, as an argument on the 
stack, or implicitly, as a part of the environment.

Andrew suggests to pass the state explicitly (as an argument in the 
stack). But taking into account that it's quite rare cases when you need 
to perform ones or other semantics independently of STATE, it's more 
simple to rely on STATE (i.e., to pass the state implicitly), and 
provide additional methods like EXECUTE-INTERPRETING and 
EXECUTE-COMPILING, or EXECUTE-IN-STATE ( i*x xt state -- j*x ) for those 
rare cases.




> The words used to use the system are:
> 
> SET-RECOGNIZER  ( xt wid -- ) Sets the wordlist execution token to a 
> recognizer execution token
> 
> DEFAULT-WIDXT  ( -- xt )   Returns the execution token of a word that 
> carries out the text interpreter's search of a wordlist
> 
> REC-NUM, REC-DNUM, REC-STR, REC-NULL name tokens to be returned to the 
> text interpreter.
> 
> An additional word GET-RECOGNIZER ( wid -- xt ) would probably be 
> desirable.
> 
> THe advantages over the RfD proposal are:
>     - simpler to understand, implement and use
>     - smaller API
>     - gives the user more flexibility about where to place recognizers
>     - provides a more general capability - not just for recognizers


Let "perceptor" is the recognizer that the Forth text interpreter 
currently uses. In any API variant and any implementation approach, we 
can have a *single* word that performs the perceptor.

In your variant this word looks through the search order. So the words 
like FIND, FIND-NAME or something alike are not used at all in your text 
interpreter.

In RfD v4 this word looks through the system recognizer sequence.

But both variants are excessive. You can define any variant in a 
portable way if you have API to set the perceptor. And it is what is 
suggested in the variant (4) in your list above.




> The above and Andrew's proposal is more about providing a way for a user 
> to define recognizers rather than defining recognizers per se. Perhaps 
> it's easier to get a consensus for that instead of a proposal such as 
> the RfD.

> Could recognizers such as in the RfD be implemented behind such 
> an interface e.g. by having them in a recognizer only wordlist at the 
> head of the search order?

Yes, it can, with some reservations, and too awkward.
At the first,  (4) can be implemented over your API, discarding all your 
functionality. And after that any other variant can be implemented over (4).

To define (4) over you API, we need to redefined the Search order word 
set to always keep a special dummy word list at the top.

   \ A dummy wordlist to handle the perceptor
   wordlist constant perceptor-wl

   \ init
   get-order perceptor-wl swap 1+ set-order

   \ The slot for the perceptor.
   \ Actually it's just a cache, since you don't provide
   \ a method to obtain a recognizer from a word list.
   variable a-perceptor

   : perceptor       ( -- xt ) a-perceptor @ ;
   : set-perceptor   ( xt -- )
     dup a-perceptor ! perceptor-wl set-recognizer
   ;
   : perceive        ( c-addr u -- i*x tt ) perceptor execute ;

   \ Redefine the standard words concerning the search order
   \ to keep perceptor-wl as a hidden item at the top.
   : (+perceptor-wl) ( -- ) get-order perceptor-wl swap 1+ set-order ;
   : words           ( -- )
     get-order nip 1- set-order words (+perceptor-wl)
   ;
   : definitions     ( -- )
     get-order 1- rot set-current ndrop
   ;
   : only            ( -- )
     only (+perceptor-wl)
   ;
   : get-order       ( -- i*x i ) get-order nip 1- ; \ hide the top
   : set-order       ( i*x i -- )  \ keep the top
       set-order (+perceptor-wl)
   ;
   : previous        ( -- ) get-order nip 1- set-order ;
   : also            ( -- ) get-order >r dup r> 1+ set-order ;
   : vocabulary      ( "name" -- )
       wordlist create , does> @ >r get-order nip r> swap set-order
   ;
   \ ORDER can be also redefined


That's all.

NB: if (4) has a DEFERRED in its API (as Bernd suggested), it cannot be 
implemented over your API.


If a more basic API can be implemented over another API, I think, the 
more basic API should be rather standardized than another one.

--
Ruvim