Drunk & Root@sh.itjust.works to Selfhosted@lemmy.worldEnglish · 1 day agoHow to combat large amounts of Ai scrapersmessage-squaremessage-square27fedilinkarrow-up180arrow-down13file-text
arrow-up177arrow-down1message-squareHow to combat large amounts of Ai scrapersDrunk & Root@sh.itjust.works to Selfhosted@lemmy.worldEnglish · 1 day agomessage-square27fedilinkfile-text
everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions
minus-squareSayCyberOnceMore@feddit.uklinkfedilinkEnglisharrow-up5·12 hours agoIf you’re able to, use GeoIP ranges to only allow access from the countries you want. That immediately limits a lot of everything Then - again if you’re able to - use a block list that covers known scrapers in case they’re in your country. I use pfBlockerNG on my pfSense firewall for exactly this.
If you’re able to, use GeoIP ranges to only allow access from the countries you want.
That immediately limits a lot of everything
Then - again if you’re able to - use a block list that covers known scrapers in case they’re in your country.
I use pfBlockerNG on my pfSense firewall for exactly this.