Author: S.Sanu (29 Dec 15 1:07am)
I am launching a new website with a database with a lot of painstakingly collected data. I don't want this to be crawled and scraped though I do want legitimate crawlers like Google.
It seems like waiting for the email to be spammed could take a long time and they would collect a lot of data from the site. Is there any pro-active way to block. Using currently blocked IPs may not work either. I am looking for honey-trap solutions that lead to hidden pages e.g. that can then be automatically IP detected and the spider blocked instantly. Do this project help in that. Sorry, I'm a newbie here ...