Message Board

http:BL Use/Development

Older Posts ]   [ Newer Posts ]
Author: S.Enbom   (30 Apr 07 3:40pm)
Whitelisting is needed, and a mechanism so that if a real humanbeing ends up banned he can solve a captcha and enter a comment which would end up on the P.H.P IP comment page.

I've been banning P.H.P caught harvesters from my site ever since P.H.P started giving out the data in RSS format so I have some experience. I very quickly got a mail from someone in Saudiarabia who could not access my site because he was using a very popular proxy that thousands, or did he say millions have to use.

Here is a quick and dirty list of IPs I've had to whitelist:
 Re: Whitelisting
Author: S.Enbom   (30 Apr 07 4:21pm)
Definately needed. Just noticed that my own website, which is hosted on a webhotel in Estonia is classified as a spamharvester after 2 visits & 1 message 6 months ago.
 Re: Whitelisting
Author: M.Janssen   (30 Apr 07 4:36pm)
The problem with whitelisting is that it's very easy for spammers to have their IP removed from the list so that they can continue spamming again. We'd have to look at how other BLs handle whitelisting and come up with a good strategy.
 Re: Whitelisting
Author: M.Prince   (30 Apr 07 4:49pm)
I'm not sure whether it is good for us to be white listing proxies like the Saudi example described above or not. Generally, I think that's a decision a website admin should make on their own site. As a result, we are currently recommending that implementations of http:BL white list at the local level. Here's how we implement it on the Apache module:

- Web admins can set a threshold of a threat score under which users are routed to a challenge page.
- The challenge page is user definable and can be a simple Javascript redirect (which bots will often fail) or a more complex CAPTCHA (e.g., type the following squiggly type, add these numbers together, identify which picture is a kitten, etc.).
- If the challenge is passed, the IP is added to a local white list and remains there for some period of time (web admin definable).
- The current version of the module does not relay this white list information back to Project Honey Pot, but we may build that functionality in at some point in the future.

One of the things we like about this type of white listing, and why we're encouraging it, is that good guys who don't know their computer has been taken over by a virus and been used to send spam can get a notice with instructions on what to do about it. For more information on why this can be a powerful tool to clean up the net, check out this article that was just written over at

There is a place for a global white list, but we need to think carefully before we implement it. in order to 1) make sure it is not abused, and 2) make sure it doesn't create an overwhelming administrative burden. For now, remember that just because an IP is listed does not necessarily mean it belongs to a spammer.

Finally, do note that IPs automatically will fall off the http:BL list if no suspicious behavior has been noted for a period of time. This helps ensure the data remains fresh.

 Re: Whitelisting
Author: J.Reid2   (12 Mar 09 10:54am)
Has anything moved forward with the whitelisting issue?

The reason I ask is, is being blocked by PHP. The only problem is this IP is registered to Google/Feedburner. So anyone using FB to monitor their rss feed's stats and PHP suddenly finds their feed no longer updates.

I can understand being cautious with system-wide whitelists, but if the IP is a known asset for a clearly reputable entity--and one that arguably won't disappear any time soon, then I think it's reasonable to make an exception.

Someone please advise.

 Re: Whitelisting
Author: J.Yard2   (16 Mar 09 3:20am)
Not everyone shares your view of Google/Feedburner being a "clearly reputable entity".
 Re: Whitelisting
Author: M.Prince   (17 Mar 09 11:05pm)
Two things:

1) Whitelisting is now available. If an IP is listed its owner can whitelist it by visiting the listing page from the IP in question (or another IP in the same network range). Whitelisting requests propagate out to the http:BL at regular intervals. Every time an IP that has been whitelisted engages in malicious behavior, the amount of time before whitelisting is allowed increases. We're experimenting with these intervals to see what works best, but so far the setup seems to be working pretty well.

2) While whitelisting on the global level is now available, we would encourage everyone who implement http:BL systems to allow for local whitelisting. This lets individual users choose whether services like Feedburner should be allowed on their own sites. If your current http:BL implementation doesn't support a local whitelist, please contact the developer and request that he or she include the feature.
 Re: Whitelisting
Author: A.Kelman   (16 Dec 09 4:15am)
Could someone explain Whitelisting? I have guys from South Africa with the same IP address and have no idea whether they are spammers on the IP address sytem their is whacky.

I notice that the IP address is close to three you white listed:

A new guy registers, Luc Hosten but he has the same IP address that I have banned previously. However, he wrote back and says he is a real person!

chantal [Find Posts by User] [View Other IP Addresses for this User]
Drukkie [Find Posts by User] [View Other IP Addresses for this User]
GinaAbles [Find Posts by User] [View Other IP Addresses for this User]
Marius [Find Posts by User] [View Other IP Addresses for this User]
Olav Hector [Find Posts by User] [View Other IP Addresses for this User]
Pieter Willering [Find Posts by User] [View Other IP Addresses for this User]
Tony Mitchell [Find Posts by User] [View Other IP Addresses for this User]

Sow how do I really know? If he has no access to actual email addresses what harm can he do in vBulletin forum?



Post Edited (16 Dec 09 4:18am)
 Re: Whitelisting
Author: A.Kelman   (17 Dec 09 1:39am)
Anyone who can reply?
 Re: Whitelisting
Author: M.Prince   (18 Dec 09 1:36am)
Not sure how to respond to your question. The "damage" a rogue user could do to your vBulletin forum, or any other web application, depends on the privileges granted. Generally I wouldn't allow any random user do anything to your web app that you cannot undo.

It seems like, in this case, the user has done quite a bit to prove that he is a human. That said, increasingly comment spammers are going through extensive steps to get onto forums and then, once on, automate the spamming.

At Project Honey Pot, we allow the owner of an IP to whitelist it by entering a CAPTCHA. If the IP falls back onto the blacklist then the process becomes increasingly more difficult. To date, it does not appear spammers are systematically whitelisting IPs. Those IPs that are whitelisted and then go back on the blacklist usually belong to users whose computers have been compromised by a virus and are being used, without their knowledge, at part of a botnet. Usually, after a few cycles, they update their anti-virus software and get their machines cleaned up.
 Re: Whitelisting
Author: J.Bro   (14 Nov 10 9:03am)

Hi folks,
I just set up my honeypot and http:BL this weekend but I'm a little confused..

Would somebody explain (or point me to some documentation)
that explains how to implement WHITE (or black) listing ??

Maybe I'm blind, but I see nothing on the site that would explain these things.
I've got http:BL running and it seems to be working, but I have no idea how to control it,
nor how to interpret the voluminous logs its writes to my error_logs -- nor what the 127.n.n.n codes coming back from signify.

If there is complete documentation somewhere, please tell me where!!


 Re: Whitelisting
Author: S.Doikas   (20 Dec 12 12:58am)
The issue with feedburner persists, any way to resolve this?
 Re: Whitelisting
Author: H.User1325   (20 Dec 12 10:54pm)
Seems to me that M.Prince responded back on 17 Mar 09 11:05pm.

When I look up with http:BL I get: a threat score of 41 with Suspicious Harvester Comment Spammer activity.

Now I don't know what your application is but I don't want anyone, Google or not, creating an account on my forums with that rating. Perhaps Prince's second suggestion, a local whitelist, is something that would solve you problem.

With a little though I wonder how http:BL is being used. Sounds to me like BL has been implemented in a way to check "any" access. Perhaps the problem is not with BL but the way it is implemented. Just a thought.

do not follow this link

Privacy Policy | Terms of Use | About Project Honey Pot | FAQ | Cloudflare Site Protection | Contact Us

Copyright © 2004–17, Unspam Technologies, Inc. All rights reserved.

contact | wiki | email