Message Board

Bugs & Development

proxy http headers

Author: B.Engert (15 Feb 05 11:43pm)

I've been looking around for a diffrent use on how to detect proxy's and I stumbled across some headers that might be of use. form the sounds of it X-forwarded-for is used by most proxy's and will tell you the ip of the user behind the proxy :-D. while some proxy's can be configured so they don't add any headers my bet is that if some one configured a open proxy they will have other settings on their defaults.

via
Forwarded
X-forwarded-for
Client-ip

Re: proxy http headers

Author: M.Prince (16 Feb 05 12:58am)

Yeah, the problem is that we've found a number of harvesters that spoof x-forwarded-for. As a result, if we rely on it if it's present then our data will get messed up.

What I think we're planning to do is during the validation process we'll visit with our own spider. We'll see whether REMOTE_HOST or X_FORWARDED_FOR is passing back the correct information. We'll then rely on whichever is authoritative.

In order to make it work, we need to make changes not only to the scripts, but also to our backend process. As soon as we do I'll post something here. We're in the midst of about a million other little things, but this is definitely in the queue.

Thanks for the info and feedback!