3dcadminA to Selfhosted@lemmy.worldEnglish · 2 days agoCloudflare blocking AI crawlerslemmy.relayeasy.comexternal-linkmessage-square33linkfedilinkarrow-up1348arrow-down12file-text
arrow-up1346arrow-down1external-linkCloudflare blocking AI crawlerslemmy.relayeasy.com3dcadminA to Selfhosted@lemmy.worldEnglish · 2 days agomessage-square33linkfedilinkfile-text
Cloudflare trying to stop AI crawling somehow! https://arstechnica.com/tech-policy/2025/07/pay-up-or-stop-scraping-cloudflare-program-charges-bots-for-each-crawl/
minus-squareProntomomo@lemmy.worldcakelinkfedilinkEnglisharrow-up5·2 days agoI don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling
I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling