Message Board

http:BL Use/Development

Older Posts ]   [ Newer Posts ]
 Getting and compiling mod_httpbl (for Apache)
Author: P.Gregg   (16 Mar 10 8:23pm)
I posted this earlier and thought I'd share (seems others had less luck in getting this running).

First you need a "cvs" client. Use your favourite package installer to install cvs for your OS. You will also need a compiler / development tools, usually this is "gcc".

With the cvs client, the following commands are what you can use to download the source code repo...

cvs -d:pserver:anonymous@httpbl.cvs.sourceforge.net:/cvsroot/httpbl login
It will ask for a password - just hit enter (for no password).

cvs -z3 -d:pserver:anonymous@httpbl.cvs.sourceforge.net:/cvsroot/httpbl co -P mod_httpbl_for_apache_2.0
(change the 2.0 to 1.3 if you are using Apache 1.3)

Your local directory will then (after several seconds) have a directory called mod_httpbl_for_apache_2.0 with the source code inside.

cd mod_httpbl_for_apache_2.0/mod_httpbl_source

You'll see a install.txt file that tells you to run: $ apxs -c -i -a mod_httpbl.c

In my case this failed to compile, complaining about missing include files, specifically pcre.h (which I know came with my PHP installation). You may need to find your pcre.h (or other include file that it complains about).

To find pcre.h I ran: find /usr -name pcre.h
In my case this came back with: /usr/local/include/php/ext/pcre/pcrelib/pcre.h

Adding this path into the apxs build command, it then looks like:

/usr/local/apache2/bin/apxs -c -i -I /usr/local/include/php/ext/pcre/pcrelib/ -a mod_httpbl.c

(you may or may not have to specify the full path to apxs, I do).

The module compiled reasonably cleanly and installed itself to my apache/modules directory. From there, follow the rest of the install.txt to add the module configuration and your HTTP:BL access key. Restart apache and you should be good to go.

The above apxs step may fail for other missing dependencies... you should be able to iterate over what it finds to be missing, extending the apxs command with additional -I paths. You may even have to install some missing libraries (such as pcre if you don't already have it).

Hope this helps someone.

Regards,
Paul Gregg www.pgregg.com
 
 Re: Getting and compiling mod_httpbl (for Apache)
Author: P.Gregg   (18 Mar 10 8:21pm)
OK - don't use the above. I found the module was causing multiple segfaults in my apache httpd.

I've abandoned trying to use this apache module - because of this and due to me realising that we turned off DNS in web servers 15 years ago for a reason* - I'm going a different approach.

I'm now writing a perl script to read in my apache log and parse out the IPs (and potentially other data), run the IP through the HTTP:BL API and *if* it comes back as suspicious I simply print it out to another logfile I call httpbl_access_log, e.g.
[ApacheDatetime] HTTPBL: IP Score HTTP:BL_lookup Type
[18/Mar/2010:23:05:17 +0000] HTTPBL: 61.149.189.29 10 127.81.12.1 Suspicious
[18/Mar/2010:23:48:17 +0000] HTTPBL: 212.235.107.11 47 127.3.47.5 Comment_Spammer,Suspicious
[18/Mar/2010:23:52:26 +0000] HTTPBL: 202.191.127.134 8 127.8.8.1 Suspicious
[18/Mar/2010:23:54:53 +0000] HTTPBL: 61.155.106.14 15 127.23.16.5 Comment_Spammer,Suspicious
...

I then setup fail2ban filter on this file to automatically firewall (iptables) these IPs for 24 hours

My firewall rule after having this running for a couple of hours.
Chain fail2ban-httpbl-access-log (1 references)
target prot opt source destination
DROP all -- 61.149.189.29 0.0.0.0/0
DROP all -- 64.214.191.210 0.0.0.0/0
DROP all -- 74.118.193.98 0.0.0.0/0
DROP all -- 201.200.221.114 0.0.0.0/0
DROP all -- 91.201.64.51 0.0.0.0/0
DROP all -- 66.90.104.159 0.0.0.0/0
DROP all -- 208.53.170.49 0.0.0.0/0
DROP all -- 174.129.116.210 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0

If there is interest, I can fix up the perl script to be a little more user friendly and describe how to configure fail2ban to use the output file.

* By "post-processing" the apache_logs, we aren't slowing up the response rate for normal users of the website, so web performance is not impacted. [Waiting for a HTTP:BL dns request is relatively slow]
Downside of this approach is that some comment/referrer spam will make it through to the logs.
 
 Re: Getting and compiling mod_httpbl (for Apache)
Author: M.Bishof   (8 Dec 10 7:00am)
Sounds interesting.

Because I have the same concern: Waiting for a HTTP:BL dns request is relatively slow

How is your experience with your new configuration?
 
 Re: Getting and compiling mod_httpbl (for Apache)
Author: H.User5946   (8 Dec 10 12:27pm)
For info - this is P.Gregg posting - I don't know why the board has screwed up my posting name.

fail2ban is relatively slow at reading the logs - sometimes taking several minutes to process.

I came up with another method using mod_perl and hooking directly into Apache - which is working excellently. My only concern with releasing the code to the world is about concerns I have posted on these boards *and* emailed to the honeypot people (with zero response) about the quality of the http:BL API database. I've found several instances of IPs which are blocked via API, but the webpage of that IP says they were last seen over a year ago (and should no longer be a threat). The lack of any official response suggests to me that Project Honey Pot is not an actively maintained, supported or encouraged project :(
http://www.projecthoneypot.org/board/read.php?f=10&i=322&t=322

I've documented what I've done on my blog:

http://www.pgregg.com/blog/2010/11/referrer-and-comment-spammers-are-a-pita.html

http://www.pgregg.com/blog/2010/12/stage-2-httpbl-with-apache2-mod-perl.html

Comments and suggestions are very welcome.

PS. It also supported flexible IP range and CIDR based white and blacklists as well as pattern matching against the referrer header.

Post Edited (8 Dec 10 12:50pm)



do not follow this link

Privacy Policy | Terms of Use | About Project Honey Pot | FAQ | Cloudflare Site Protection | Contact Us

Copyright © 2004–24, Unspam Technologies, Inc. All rights reserved.

contact | wiki | email