Path: ...!3.eu.feeder.erje.net!2.eu.feeder.erje.net!feeder.erje.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Lawrence D'Oliveiro <ldo@nz.invalid>
Newsgroups: comp.os.linux.advocacy
Subject: Re: More Funny Stuff From Python
Date: Sun, 16 Jun 2024 01:02:05 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 49
Message-ID: <v4ldic$3n6db$1@dont-email.me>
References: <17d716103c089ab3$7951$675878$802601b3@news.usenetexpress.com>
	<ej496jd0tb59u2l0nqtghq3u9ahhmann3s@4ax.com>
	<lcjnnuF896hU5@mid.individual.net>
	<kma96j1no1tp926ctejldkdk2c19aeruft@4ax.com>
	<lcjvk1F9n7aU1@mid.individual.net>
	<2ej96j1mbvgiok4q5c57vdlo94itpfu5dt@4ax.com>
	<6664e989$0$2363151$882e4bbb@reader.netnews.com>
	<slrnv6f9fl.2hg.candycanearter07@candydeb.host.invalid>
	<66687931$0$3747328$882e4bbb@reader.netnews.com>
	<slrnv6ig9v.13pi.candycanearter07@candydeb.host.invalid>
	<66699f8c$0$966$882e4bbb@reader.netnews.com>
	<slrnv6l2vk.24pu.candycanearter07@candydeb.host.invalid>
	<666b0963$0$985$882e4bbb@reader.netnews.com>
	<ld0olnF7p22U1@mid.individual.net>
	<666b43c2$0$966$882e4bbb@reader.netnews.com>
	<ld19dnFa6idU1@mid.individual.net>
	<666c4979$0$2363133$882e4bbb@reader.netnews.com>
	<ld3affFja99U1@mid.individual.net>
	<666c8888$0$7063$882e4bbb@reader.netnews.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 16 Jun 2024 03:02:05 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="bd45dcd0b5a25b060647b4de82583354";
	logging-data="3905963"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18MDBC71voOdy4v6Yw0mv3b"
User-Agent: Pan/0.158 (Avdiivka; )
Cancel-Lock: sha1:t1UixXYURnCcipBMWMsDaYwnCjQ=
Bytes: 3334

On Fri, 14 Jun 2024 14:14:33 -0400, DFS wrote:

> Here's how python checks array bound every time the
> array is accessed:
> 
> 
> int PyList_SetItem(PyObject *op, Py_ssize_t i,
>                 PyObject *newitem)
> {
>      PyObject **p;
>      if (!PyList_Check(op)) {
>          Py_XDECREF(newitem); PyErr_BadInternalCall();
>          return -1;
>      }
>      if (!valid_index(i, Py_SIZE(op))) {
>          Py_XDECREF(newitem); PyErr_SetString(PyExc_IndexError,
>                          "list assignment index out of range");
>          return -1;
>      }
>      p = ((PyListObject *)op) -> ob_item + i;
>      Py_XSETREF(*p, newitem);
>      return 0;
> }
> 
> 
> Python-3.11.0.tgz.2\Python-3.11.0\Objects\listobject.c line 257
> 
> 
> Which is another thing that contributes to its slowness vs C.

From the same source file:

    static inline int
    valid_index(Py_ssize_t i, Py_ssize_t limit)
    {
        /* The cast to size_t lets us use just a single comparison
           to check whether i is in the range: 0 <= i < limit.

           See:  Section 14.2 "Bounds Checking" in the Agner Fog
           optimization manual found at:
           https://www.agner.org/optimize/optimizing_cpp.pdf
        */
        return (size_t) i < (size_t) limit;
    }

Somehow I don’t think that’s the bottleneck in PyList_SetItem.

Also have a look at the list_extend routine, since that is commonly used 
for appending lots of items.