Path: ...!news.mixmin.net!aioe.org!.POSTED!not-for-mail
From: Ruvim <ruvim.pinka@gmail.com>
Newsgroups: comp.lang.forth
Subject: Re: Announcing Recognizer RFD v4 and Call For Vote
Date: Fri, 14 Sep 2018 21:41:12 +0300
Organization: Aioe.org NNTP Server
Lines: 86
Message-ID: <pngvcc$pta$1@gioia.aioe.org>
References: <ddf864ae-d379-47be-93a1-1baaff2acc24@googlegroups.com>
 <2018Aug21.191026@mips.complang.tuwien.ac.at> <plhsln$d9a$1@dont-email.me>
 <2018Aug22.091517@mips.complang.tuwien.ac.at>
 <d983dece-80bb-46ff-9aaa-784e81cd09ed@googlegroups.com>
 <2018Aug22.144751@mips.complang.tuwien.ac.at> <pljpc8$2gd$1@dont-email.me>
 <2018Aug22.161749@mips.complang.tuwien.ac.at> <pndjtk$117m$1@gioia.aioe.org>
 <pndnfj$3vc$1@dont-email.me> <pne7i4$8eb$1@gioia.aioe.org>
 <pneeba$1no$1@dont-email.me> <pnej1n$v3l$1@dont-email.me>
 <pnftoe$ptv$1@gioia.aioe.org> <pngl8r$fn1$1@dont-email.me>
NNTP-Posting-Host: Gf+R3KkStRuWf94htD9y3Q.user.gioia.aioe.org
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Complaints-To: abuse@aioe.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101
 Thunderbird/52.9.1
Content-Language: en-US
X-Notice: Filtered by postfilter v. 0.8.3
Bytes: 4329

On 2018-09-14 18:48, Alex McDonald wrote:
> On 14-Sep-18 10:07, Ruvim wrote:
>> On 2018-09-13 23:58, Alex McDonald wrote:
[...]
> I don't see why we need in this proposal or any other to add 
> another set of concepts drawn from other disciplines or languages, and 
> applied where they seem designed to obscure the simple nature of the 
> process.

We need to name things to describe conceptions.
And there is a terminology that is common for all programming languages. 
Why don't to use it?

I agree, we should give definitions to be accurate.


=== Uncareful introduction

In Forth we have the following general steps
of source text translation:

   1. ->(parsing)-> lexemes
   2. ->(resolving)-> tokens
   3. ->(token-translation)-> side-effects

Where
  parsing (1) is going via PARSE-NAME and PARSE words.
  resolving (2) is going via FIND-NAME and RESOLVE-LEXEME in general case
  token-translation (3) is going via TRANSLATE-TOKEN word

Parsing is syntax analysis.
Resolving (and names resolution) is semantic analysis.
Translation is changing form in general.

Linking with Section 3.4 "The Forth text interpreter" of the Forth-2012 
standard:
   (1) is item (a)
   (2) is item (b) and item (c) (without sub-items).
   (3) is items (b.1-2) and (c.1-2).



==== Questions and Answers.


- Why do we need the new terms?
Because we want to generalize the items (b) and (c) of the Forth text 
interpreter. Hence we need to name this conception.
The "resolving" and *resolver* are appropriate terms.

- What does a resolver resolves?
It resolves names, numbers, literals, etc that are sequences of 
characters. We need to name them as well. "Lexeme" is a good choice that 
is relevant for our topic. So, a resolver resolves *lexemes*.


- What does a resolver produce?
It produces tokens. We already have "name token" and "execution token". 
So, for generalization, any successful output form resolver
can be called "token". And we will get "single-cell number token", 
"double-cell number token", "string token", etc.

We also want to have chains of resolvers. To know what token is returned 
(is it number, or xt, or string), a resolver should also return a token 
type for that. Each token type should be a unique number.

So, a resolver produces a *token* and a *token type*.


- Why should we know a token type?
For tokens of different types we should use different methods of 
translation (see (b.1-2) and (c.1-2) items of the Forth text 
interpreter). And therefore each token type should be associated with 
the corresponding method of translation.

Yes, *translation* is yet another new term. But we need to name the 
general conception of the (b.1-2) and (c.1-2) items.


- How is a token type associated with a method of translation?
Well. Why don't to to use the xt of a token translation method as a 
token type?  Hm.. why not?  It is an elegant solution.


--
Ruvim