Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Lawrence D'Oliveiro <ldo@nz.invalid>
Newsgroups: comp.lang.c
Subject: Re: program to remove duplicates
Date: Sun, 22 Sep 2024 02:06:49 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 10
Message-ID: <vcnu3p$1vkui$2@dont-email.me>
References: <ecb505e80df00f96c99d813c534177115f3d2b15@i2pn2.org>
	<vcnfbi$1ocq6$1@dont-email.me>
	<8630bec343aec589a6cdc42bb19dae28120ceabf@i2pn2.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 22 Sep 2024 04:06:50 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="f08cca55e0d941314925e901504a6690";
	logging-data="2085842"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/Lcsck8I1Xics/1R2WGARZ"
User-Agent: Pan/0.160 (Toresk; )
Cancel-Lock: sha1:hZSNAu2FCCkpj8pAuZIJHnEi43k=
Bytes: 1311

On Sun, 22 Sep 2024 00:18:09 +0200, fir wrote:

> ... you just need to read all files in 
> folder and compare it byte by byte to other files in folder of the same
> size

For N files, that requires N × (N - 1) ÷ 2 byte-by-byte comparisons. 
That’s an O(N²) algorithm.

There is a faster way.