Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: David Brown <david.brown@hesbynett.no>
Newsgroups: comp.lang.c
Subject: Re: realloc() - frequency, conditions, or experiences about
 relocation?
Date: Mon, 17 Jun 2024 14:15:11 +0200
Organization: A noiseless patient Spider
Lines: 32
Message-ID: <v4p9cg$lbo4$1@dont-email.me>
References: <v4ojs8$gvji$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Mon, 17 Jun 2024 14:15:12 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="6eee6dfb82180fb756db1a7758fc4b5a";
	logging-data="700164"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18hoQkLiBqUX7MJ+stryC9e4Fy003XgYyY="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Cancel-Lock: sha1:49/onGY4mU4/oKMULHEjs17B9xY=
Content-Language: en-GB
In-Reply-To: <v4ojs8$gvji$1@dont-email.me>
Bytes: 2788

On 17/06/2024 08:08, Janis Papanagnou wrote:
> In a recent thread realloc() was a substantial part of the discussion.
> "Occasionally" the increased data storage will be relocated along
> with the previously stored data. On huge data sets that might be a
> performance factor. Is there any experience or are there any concrete
> factors about the conditions when this relocation happens? - I could
> imagine that it's no issue as long as you're in some kB buffer range,
> but if, say, we're using realloc() to substantially increase buffers
> often it might be an issue to consider. It would be good to get some
> feeling about that internal.
> 
> Janis

Consider your target audience and their hardware, the target OS, and the 
realistic size of your data.  If the target is a PC, you can happily 
malloc tens of MB at the start without a care, and for systems that do 
not actually allocate system memory until you try to access the area, 
there is no cost to this.

So in many situations where you are reading and parsing data from a 
file, you can just do the initial malloc with more than enough space for 
any realistic input file.  You might still implement a realloc solution 
for occasional extreme uses, and because it is nice to avoid artificial 
limits for programs, but efficiency matters a lot less in those cases.

It may also be the case that even if realloc returns a different address 
and logically copies a lot of data, that this is done by smarter virtual 
memory mapping so that only the mapping changes, and the underlying 
physical ram does not need to be copied.  But I don't know if OS's and 
realloc implementations are smart enough to do that.