Message Board

Installing Honey Pots

Proxy servers?

Author: L.Skjaerlund (18 Jan 05 2:28am)

From reading the FAQ it appears that Honey Pot wants the IPs of harvesters. I suppose it does this by linking the handed-out email address to the IP of the "visitor"?

However, what happens when our webservers are protected by a proxy servers - meaning that all requests are coming from this proxy IP with the only meaningful information beeing in the proxy servers logs?

Regards,
Lars

Re: Proxy servers?

Author: M.Prince (18 Jan 05 4:14am)

We anticipate that, over time, harvesters like spam servers will move to using proxy servers in order to obscure their identities. When this happens -- and we've seen some indication that with at least a few harvesters it's happening already -- it will be harder to use harvester data in order to determine spammers identities. We're encouraged that so few harvesters today are using proxies to obscure their identities. We recognize that as soon as we start going after them, this is bound to change. However, the data from harvester traffic, even if through proxies, is still useful. For example, website administrators can choose to block the traffic generated from known proxies in order to decrease the risk to their sites.

While we don't know what conclusions the data will lead us to eventually, we do know that, to this point, no one was gathering information on the first stage of the spam cycle. Project Honey Pot has begun to gather that information and, as such, we hope will be useful to the overall spam fight.

Re: Proxy servers?

Author: L.Skjaerlund (18 Jan 05 11:34am)

I'm afraid I didn't make myself clear: I'm not talking about spammers hiding behind proxies (though I do see that problem as well), I'm talking about our own servers beeing protected behind a reverse proxy.

So, my question is: Would it make any sense to install Honey Pot pages on our webservers as you wouldn't be able to obtain the IP of the harvester?

_Not_ because the harvester is hiding behind a proxy, but because our webservers are hiding behind a reverse proxy.

Re: Proxy servers?

Author: M.Prince (19 Jan 05 12:17am)

Gotcha. Sorry for the misunderstanding. I understand now.

Right now our scripts are configured to pickup the Remote Address (the IP that connects with the script). You are correct that in your case that would result in your proxy being listed as the harvester. As a result, in your case it doesn't make sense to install a honey pot.

There may be a solution. We can watch for the X-Forwarded-For variable. Proxies are typically configured to put the connecting IP address in the X-Forwarded-For environment variable. By reading that, therefore, we may be able to retrieve the harvester's IP even when you're behind a proxy.

In fact this was our original plan. When we first launched the Project we actually reported that variable if it was present in the server environment, instead of the Remote Address. However, we found that a surprising number of harvesters actually forged the X-Forwarded-For information.

If there are a number of users with web servers behind proxies then maybe we'll create special "proxy" versions of the script. Or maybe we can use some trick where if the Remote Address known to be your legitimate proxy then we will trust the X-Forwarded-For info. Let me think if there's a way to make it work.

In the meantime, if there are other folks who are trying to install a honey pot on a server behind a proxy, post here and let us know. Maybe let us know what your preferred scripting platform is as well.....

Thanks for the feedback. I'll post here if we come up with a solution.

Post Edited (18 Jan 05 11:28pm)

Re: Proxy servers?

Author: M.Healan (27 Jan 05 8:43pm)

This may effect me at various times in the future. My site is prone to being attacked by armies of DDoS bots (I tend to tick off unethical people). When I am under attack (or experiencing a slashdotting), I have to switch to routing through several rotating proxy servers to keep the site going.

I'm not even sure how to set them to forward the user's IP but I'm sure I could figure it out if I had to.

Re: Proxy servers?

Author: C.Kruslicky (6 Feb 05 10:57am)

One setup I come across sometimes is an older Apache reverse proxy, handling static content while proxying requests for perl scripts to an internal server. In these cases I believe the remote ip can be had on the internal server like this: $ENV{'HTTP_X_FORWARDED_FOR'}

Re: Proxy servers?

Author: M.Prince (6 Feb 05 3:21pm)

We originally pulled the HTTP_X_FORWARDED_FOR environmental variable in order to catch the reverse proxy problem. In fact, the algorithm we relied upon was something like:

if (HTTP_X_FORWARDED_FOR is present) then use that;
else use REMOTE_ADDRESS.

Problem was that we found a BUNCH of harvesters were inserting a phony HTTP_X_FORWARDED_FOR variable. As a result, we were getting bad data whenever one of those harvesters came through.

We've toyed with a couple ways to fix this, but they all involve pretty substantial changes, or risk user confusion. The first would be to provide a version of the script for folks behind reverse proxies. Maybe have a check box that says, "Are you behind a reverse proxy? If so, click here." I think that solution is out because, as you can imagine, it would confuse about 85% of our users.

The better solution would involve some work on our part. What we'd do is report back both the REMOTE_ADDRESS and HTTP_X_FORWARDED_FOR. When a honey pot is activated we visit it with our own sider. We know our spider's IP address. We would then watch to see which, if either, variable was reporting the correct IP and then rely on that for our data.

That would require a change in all the scripts (to report the additional variable) as well as some back end coding on our part for the logic in terms of which variable to rely on for which honey pots. It's something we've thought about, but probably won't be implemented until at least we roll out v.0.2 of the pots.... probably in a few months.

If anyone has a more elegant solution, let us know and we'll get it up and running sooner!