Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Uwe Schmitz <schmitzu@mail.de>
Newsgroups: comp.lang.tcl
Subject: Re: Tcl9: source files are interpreted as utf-8 by default
Date: Tue, 7 Jan 2025 18:00:15 +0100
Organization: A noiseless patient Spider
Lines: 56
Message-ID: <vljmj2$29vdo$1@dont-email.me>
References: <vjhiar$3f9go$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 07 Jan 2025 18:00:19 +0100 (CET)
Injection-Info: dont-email.me; posting-host="3511bd66db0b1206656c0f9efda30f8a";
	logging-data="2424248"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX19pb5qMyvxTG9/NIOfnyHwFFbAb0vgCx+0="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:+LkN3qJi8Qfv1+zbrrsaeWl8AlA=
Content-Language: de-DE
In-Reply-To: <vjhiar$3f9go$1@dont-email.me>
Bytes: 2952

Sorry that I have to come back to this issue.

As stated before, we use iso8859-1 as system encoding.
With Tcl9 we now got errors reading source files with e.g. umlauts,
because Tcl9 interprets all sources as utf-8 by
default. That means we have to add "-encoding iso8859-1"
to ALL source and ALL tclsh calls in ALL scripts.
So far, so good(or bad?).

What initially seems quite doable, looks more and more scary
to me. First, if we ever may switch encoding to utf-8 we
have to alter all those lines again. Either we switch them to utf-8 or
we remove the -encoding and went back
to the state before Tcl9.

Another point: we have MANY scripts only for development needs.
Coded quickNdirty for code generation, documentation, packaging, etc.
Most of them called by "tclsh helperScript.tcl ..." (they have no shebang or
whatever). They now have to be called by
"tclsh -encoding iso8859-1 helperScript.tcl ..."

Thats a lot more typing.

Some of them have a usage message like:
usage: tclsh helperScript.tcl arg1 arg2
....

Do we now have to change it to:
usage: tclsh -encoding iso8859-1 helperScript.tcl arg1 arg2
....
?

Side note: The open command, which opens a file with the
system encoding by default, has thankfully not changed
in the same manner as source and tclsh :-).

Now my suggestion:
Wouldn't it be convenient for Tcl9 to have a global switch
(e.g. Environment variable) to get back the Tcl8 encoding
behaviour?
Or, isn't it best to keep the old encoding mimik in Tcl9.
I see no advantage in the new behavior. Even if you have
all sources in utf-8, you might also have choosen utf-8 as
system encoding.

Do we have (or can we have) a magic comment or something else
with which we can choose the encoding of a source file in
the file itself?
Python e.g. has https://peps.python.org/pep-0263/.

Maybe I’m missing something crucial...

Thanks in advance
Uwe