Path: ...!weretis.net!feeder9.news.weretis.net!news.quux.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Mikko <mikko.levanto@iki.fi>
Newsgroups: comp.theory
Subject: Re: neos Universal Compiler
Date: Wed, 19 Mar 2025 18:11:18 +0200
Organization: -
Lines: 69
Message-ID: <vreqb6$174ma$1@dont-email.me>
References: <UX6BP.512735$Kb9a.408485@fx16.ams4> <vr3ir8$39jtg$1@dont-email.me> <3KgBP.513160$Kb9a.94584@fx16.ams4> <vr6948$1jnlo$1@dont-email.me> <KWEBP.424367$NN2a.82900@fx15.ams4> <vr8pi1$3r8lv$1@dont-email.me> <NrYBP.191725$C8m7.108955@fx11.ams4> <vrbu8h$2km30$1@dont-email.me> <S7fCP.743699$nb1.627959@fx01.ams4> <vre64b$lcu3$1@dont-email.me> <2AACP.939386$gHk7.592478@fx17.ams4>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 19 Mar 2025 17:11:19 +0100 (CET)
Injection-Info: dont-email.me; posting-host="ad2e39e96e6195845187b0d14619ffc0";
	logging-data="1282762"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/R7G8pelR3j0/sbGU9AB3F"
User-Agent: Unison/2.2
Cancel-Lock: sha1:0SIyT/rK1dloV4r06uJm+56k7r0=
Bytes: 3514

On 2025-03-19 14:32:30 +0000, Mr Flibble said:

> On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:
> 
>> On 2025-03-18 14:08:50 +0000, Mr Flibble said:
>> 
>>> On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:
>>> 
>>>> On 2025-03-17 16:53:01 +0000, Mr Flibble said:
>>>> 
>>>>> On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:
>>>>> 
>>>>>> On 2025-03-16 18:40:42 +0000, Mr Flibble said:
>>>>>> 
>>>>>>> On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:
>>>>>>> 
>>>>>>>> On 2025-03-15 15:08:47 +0000, Mr Flibble said:
>>>>>>>> 
>>>>>>>>> On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:
>>>>>>>>> 
>>>>>>>>>> On 2025-03-15 04:00:52 +0000, Mr Flibble said:
>>>>>>>>>> 
>>>>>>>>>>> Hi!
>>>>>>>>>>> 
>>>>>>>>>>> neos universal compiler (that can compile any programming
>>>>>>>>>>> language)
>>>>>>>>>>> is successfully running the tokenization stage tokenizing a
>>>>>>>>>>> program written in the neos reference language. #cpp #coding
>>>>>>>>>>> #compiler #compsci #gamedev
>>>>>>>>>>> 
>>>>>>>>>>> /Flibble
>>>>>>>>>> 
>>>>>>>>>> Can it tokenize FORTRAN 60 or FORTRAN IV ?
>>>>>>>>> 
>>>>>>>>> ANY programming language.
>>>>>>>>> 
>>>>>>>>> /Flibble
>>>>>>>> 
>>>>>>>> How is neos configured to tokenize FORTRAN 60 ?
>>>>>>> 
>>>>>>> The same way you would configure it for any other programming
>>>>>>> language.
>>>>>> 
>>>>>> If it is configured the same for each programming language then how
>>>>>> does it know how to tokenize?
>>>>> 
>>>>> You configure it by providing a language specific neosBNF schema
>>>>> (grammar)
>>>>> file (an input to the compilalation process).
>>>> 
>>>> Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
>>>> Algol 60?
>>> 
>>> Not yet.
>> 
>> The definition of string literal of Algol 60 would be a good example of
>> something that cannot be defined with a regular expression and is
>> therefore impossible or at least complicated with an ordinary tokenizer.
> 
> neos does not use regular expressions and the neos grammar is context
> sensitive (i.e. not context free).

A context free grammar should be enough for tokenizing strings of Algol 60.

Tokenizing statements of FORTRAN 66 is a harder problem.

-- 
Mikko