Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Mikko <mikko.levanto@iki.fi>
Newsgroups: comp.theory
Subject: Re: neos Universal Compiler and FORTRAN IV
Date: Fri, 28 Mar 2025 14:01:40 +0200
Organization: -
Lines: 104
Message-ID: <vs6334$2mosu$1@dont-email.me>
References: <UX6BP.512735$Kb9a.408485@fx16.ams4> <vr3ir8$39jtg$1@dont-email.me> <3KgBP.513160$Kb9a.94584@fx16.ams4> <vr6948$1jnlo$1@dont-email.me> <KWEBP.424367$NN2a.82900@fx15.ams4> <vr8pi1$3r8lv$1@dont-email.me> <NrYBP.191725$C8m7.108955@fx11.ams4> <vrbu8h$2km30$1@dont-email.me> <S7fCP.743699$nb1.627959@fx01.ams4> <vre64b$lcu3$1@dont-email.me> <2AACP.939386$gHk7.592478@fx17.ams4> <vreqb6$174ma$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Mar 2025 13:01:41 +0100 (CET)
Injection-Info: dont-email.me; posting-host="6b7417c2cac36ebd3726a4ec4446c661";
	logging-data="2843550"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/snkZFYEV0oGv4AZhf+9PZ"
User-Agent: Unison/2.2
Cancel-Lock: sha1:xqhG8yrlC1ADCdtXOdFCQpqmv0U=
Bytes: 4321

On 2025-03-19 16:11:18 +0000, Mikko said:

> On 2025-03-19 14:32:30 +0000, Mr Flibble said:
> 
>> On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:
>> 
>>> On 2025-03-18 14:08:50 +0000, Mr Flibble said:
>>> 
>>>> On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:
>>>> 
>>>>> On 2025-03-17 16:53:01 +0000, Mr Flibble said:
>>>>> 
>>>>>> On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:
>>>>>> 
>>>>>>> On 2025-03-16 18:40:42 +0000, Mr Flibble said:
>>>>>>> 
>>>>>>>> On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:
>>>>>>>> 
>>>>>>>>> On 2025-03-15 15:08:47 +0000, Mr Flibble said:
>>>>>>>>> 
>>>>>>>>>> On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:
>>>>>>>>>> 
>>>>>>>>>>> On 2025-03-15 04:00:52 +0000, Mr Flibble said:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi!
>>>>>>>>>>>> 
>>>>>>>>>>>> neos universal compiler (that can compile any programming
>>>>>>>>>>>> language)
>>>>>>>>>>>> is successfully running the tokenization stage tokenizing a
>>>>>>>>>>>> program written in the neos reference language. #cpp #coding
>>>>>>>>>>>> #compiler #compsci #gamedev
>>>>>>>>>>>> 
>>>>>>>>>>>> /Flibble
>>>>>>>>>>> 
>>>>>>>>>>> Can it tokenize FORTRAN 60 or FORTRAN IV ?
>>>>>>>>>> 
>>>>>>>>>> ANY programming language.
>>>>>>>>>> 
>>>>>>>>>> /Flibble
>>>>>>>>> 
>>>>>>>>> How is neos configured to tokenize FORTRAN 60 ?
>>>>>>>> 
>>>>>>>> The same way you would configure it for any other programming
>>>>>>>> language.
>>>>>>> 
>>>>>>> If it is configured the same for each programming language then how
>>>>>>> does it know how to tokenize?
>>>>>> 
>>>>>> You configure it by providing a language specific neosBNF schema
>>>>>> (grammar)
>>>>>> file (an input to the compilalation process).
>>>>> 
>>>>> Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
>>>>> Algol 60?
>>>> 
>>>> Not yet.
>>> 
>>> The definition of string literal of Algol 60 would be a good example of
>>> something that cannot be defined with a regular expression and is
>>> therefore impossible or at least complicated with an ordinary tokenizer.
>> 
>> neos does not use regular expressions and the neos grammar is context
>> sensitive (i.e. not context free).
> 
> A context free grammar should be enough for tokenizing strings of Algol 60.
> 
> Tokenizing statements of FORTRAN 66 is a harder problem.

Some test cases for a FORTRAN IV tokenizer:

        REALITY IS, AS I MAY
        LOGICALLY SAY, REAL
        LOGICAL = WHAT I SAY

Note that these lines begin with 8 spaces.

Tokens on the first line:
keyword REAL
identifier ITYIS
comma
identifier ASIMAY
end of statement

Tokens on the second line:
keyword LOGICAL
identifier LYSAY
comma
identifier REAL
end of statement

Tokens on the third line:
identifier LOGICAL
assignment symbol
identifier WHAT I SAY

Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.

I don't understand how neos can be configred to tokenize the above
test lines and other FORTRAN IV statements.

-- 
Mikko