Deutsch English Français Italiano |
<slrn101ueaa.198p.anthk@openbsd.home.localhost> View for Bookmarking (what is this?) Look up another Usenet article |
Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail From: anthk <anthk@openbsd.home> Newsgroups: comp.misc Subject: Re: bad bot behavior Date: Mon, 12 May 2025 06:24:46 -0000 (UTC) Organization: A noiseless patient Spider Lines: 41 Message-ID: <slrn101ueaa.198p.anthk@openbsd.home.localhost> References: <vrc2r4$2okrp$1@dont-email.me> <vrc8qm$2tkq5$1@dont-email.me> <20250318182006.00006ae3@dne3.net> <slrnvtlcpl.41d.${send-direct-email-to-news1021-at-jusme-dot-com-if@vm46.home.jusme.com> Injection-Date: Mon, 12 May 2025 08:24:46 +0200 (CEST) Injection-Info: dont-email.me; posting-host="cc7a24369ce4842ffee3a5ba377d97da"; logging-data="1014028"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/YpQdJN9CwDwCkE8jq4IyC" User-Agent: slrn/1.0.3 (OpenBSD) Cancel-Lock: sha1:l4DwnJ03DwBygNO6ov62t+3xOqw= Bytes: 2068 On 2025-03-19, Ian <${send-direct-email-to-news1021-at-jusme-dot-com-if-you-must}@jusme.com> wrote: > On 2025-03-18, Toaster <toaster@dne3.net> wrote: >> >> But what can be done to mitigate this issue? Crawlers and bots ruin the >> internet. > > #mode=evil > > How about a script that spews out an endless stream of junk from > /usr/share/dict/words, parked on a random URL that's listed in > robots.txt as forbidden. Any bot choosing to chew on that gets what > it deserves, though you might need to bandwidth limit it. > > Perl, cpanm and Hailo. Set a nonsense.txt text file with one sentence per line. Like this: rm -rf boosts performance under Ubuntu. fedora it's updated with apt-get dist-upgrade. openbsd works fine with ZFS. And so on... $>cpanm -n Hailo $>Hailo -t nonsense.txt -b output.brn Now, create a simple Perl program (really easy with Hailo and trivial input/output). Run 'perldoc Hailo' once it's installed for a quick usage guide. Redirect that outputted nonsense to a file: perl yourhailoscript.pl > crap.txt Have fun.