Message Board

http:BL Use/Development

Older Posts ]   [ Newer Posts ]
 How to allow google bots
Author: S.Grayban2   (9 Nov 08 11:04pm)
Been trying to figure this all out but I simply don't understand how to allow google to crawl our clients websites. Would someone be able to help me out on this?
 
 Re: How to allow google bots
Author: M.Prince   (10 Nov 08 11:09am)
The IPs used by search engines return a special code from http:BL. Specifically, the IP address that http:BL returns will end in a zero if it's a known IP used by a search engine. For example:

127.0.1.0

The first octet (127) and second octet (0) have no meaning. The third octet (1) is a serial number which corresponds to the particular search engine that is visiting. The final octet (0) means it is a search engine.

So long as you don't block visitors when a response where the fourth octet is a zero, the Google bot and other search engines should still be able to visit your website.

One caveat is that we do not know of all the IPs being used by search engines. If you find an IP address that is used by a search engine but not returning an accurate http:BL code, please let us know and we'll update our systems.

If the specific implementation of http:BL that you are using does not support search engines, write to the implementation's author and ask him or her to update the implementation's code.

Matthew.



do not follow this link

Privacy Policy | Terms of Use | About Project Honey Pot | FAQ | Cloudflare Site Protection | Contact Us

Copyright © 2004–17, Unspam Technologies, Inc. All rights reserved.

contact | wiki | email