Path: ...!goblin2!goblin.stu.neva.ru!aioe.org!eternal-september.org!feeder.eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail From: Ruvim Newsgroups: comp.lang.forth Subject: Re: Recognizers - an alternative Date: Sat, 24 Oct 2020 17:14:31 +0300 Organization: A noiseless patient Spider Lines: 160 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Date: Sat, 24 Oct 2020 14:14:34 -0000 (UTC) Injection-Info: reader02.eternal-september.org; posting-host="8532595ce0a0396ed06af54b584838df"; logging-data="11266"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19zsYZ0g1yi92kyR++j9vnb" User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 Cancel-Lock: sha1:Bs+N7zfp4ZpMJY5se89cc0BBHfM= In-Reply-To: Content-Language: en-US X-Mozilla-News-Host: news://nntp.aioe.org Bytes: 7155 On 2020-10-13 14:14, Gerry Jackson wrote: > There have been four recognizer proposals, one an RfD and three prior to > the September Forth 200X meeting. These are: > > 1. the official RfD > http://amforth.sourceforge.net/pr/Recognizer-rfc-D.html (this link > doesn't seem to be working at present, I don't know of another) > 2. Andrew Haley's simple recognizers > https://forth-standard.org/proposals/an-alternative-to-the-recognizer-proposal#reply-493 > > 3. Anton Ertl's Nestable recognizer sequences > https://forth-standard.org/proposals/nestable-recognizer-sequences#contribution-149 > > 4. Bernd Paysan's Minimalistic API > https://forth-standard.org/proposals/minimalistic-core-api-for-recognizers#reply-515 [...] > Of the proposals it seems to me that Andrew's  approach is better > because of its simplicity, ease of implementation and the flexibility > that it gives users when defining recognizers. > > As an experiment to see how well it works I've implemented a scheme > similar to Andrew's. I would prefer to also see the code :) Since your description leaves some parts too unclear. Gist at GitHub is a useful platform for that. https://gist.github.com/ [...] > I didn't find it necessary to pass a mode value to the recognizer as > Andrew proposed but one could easily be added. It's that since in the API of Andrew a user provides not a recognizer for lexeme, but a translator for lexeme. You don't need to know the state for recognizing. But you need to know the state for translation (i.e. to perform interpretation or compilation semantics depending on STATE). In general, the state can be passed explicitly, as an argument on the stack, or implicitly, as a part of the environment. Andrew suggests to pass the state explicitly (as an argument in the stack). But taking into account that it's quite rare cases when you need to perform ones or other semantics independently of STATE, it's more simple to rely on STATE (i.e., to pass the state implicitly), and provide additional methods like EXECUTE-INTERPRETING and EXECUTE-COMPILING, or EXECUTE-IN-STATE ( i*x xt state -- j*x ) for those rare cases. > The words used to use the system are: > > SET-RECOGNIZER  ( xt wid -- ) Sets the wordlist execution token to a > recognizer execution token > > DEFAULT-WIDXT  ( -- xt )   Returns the execution token of a word that > carries out the text interpreter's search of a wordlist > > REC-NUM, REC-DNUM, REC-STR, REC-NULL name tokens to be returned to the > text interpreter. > > An additional word GET-RECOGNIZER ( wid -- xt ) would probably be > desirable. > > THe advantages over the RfD proposal are: >    - simpler to understand, implement and use >    - smaller API >    - gives the user more flexibility about where to place recognizers >    - provides a more general capability - not just for recognizers Let "perceptor" is the recognizer that the Forth text interpreter currently uses. In any API variant and any implementation approach, we can have a *single* word that performs the perceptor. In your variant this word looks through the search order. So the words like FIND, FIND-NAME or something alike are not used at all in your text interpreter. In RfD v4 this word looks through the system recognizer sequence. But both variants are excessive. You can define any variant in a portable way if you have API to set the perceptor. And it is what is suggested in the variant (4) in your list above. > The above and Andrew's proposal is more about providing a way for a user > to define recognizers rather than defining recognizers per se. Perhaps > it's easier to get a consensus for that instead of a proposal such as > the RfD. > Could recognizers such as in the RfD be implemented behind such > an interface e.g. by having them in a recognizer only wordlist at the > head of the search order? Yes, it can, with some reservations, and too awkward. At the first, (4) can be implemented over your API, discarding all your functionality. And after that any other variant can be implemented over (4). To define (4) over you API, we need to redefined the Search order word set to always keep a special dummy word list at the top. \ A dummy wordlist to handle the perceptor wordlist constant perceptor-wl \ init get-order perceptor-wl swap 1+ set-order \ The slot for the perceptor. \ Actually it's just a cache, since you don't provide \ a method to obtain a recognizer from a word list. variable a-perceptor : perceptor ( -- xt ) a-perceptor @ ; : set-perceptor ( xt -- ) dup a-perceptor ! perceptor-wl set-recognizer ; : perceive ( c-addr u -- i*x tt ) perceptor execute ; \ Redefine the standard words concerning the search order \ to keep perceptor-wl as a hidden item at the top. : (+perceptor-wl) ( -- ) get-order perceptor-wl swap 1+ set-order ; : words ( -- ) get-order nip 1- set-order words (+perceptor-wl) ; : definitions ( -- ) get-order 1- rot set-current ndrop ; : only ( -- ) only (+perceptor-wl) ; : get-order ( -- i*x i ) get-order nip 1- ; \ hide the top : set-order ( i*x i -- ) \ keep the top set-order (+perceptor-wl) ; : previous ( -- ) get-order nip 1- set-order ; : also ( -- ) get-order >r dup r> 1+ set-order ; : vocabulary ( "name" -- ) wordlist create , does> @ >r get-order nip r> swap set-order ; \ ORDER can be also redefined That's all. NB: if (4) has a DEFERRED in its API (as Bernd suggested), it cannot be implemented over your API. If a more basic API can be implemented over another API, I think, the more basic API should be rather standardized than another one. -- Ruvim