using htaccess to block referrer spam and hotlinks

using htaccess to block referrer spam and hotlinks

Posted by: supernaut
Posted on: 2006-04-22 01:24:00

This is a two-part question I've done alot of searching and experimenting on and would really appreciate some answers I can't seem to find.

I'm using Movable Type 3.2 and get a lot of referrer spam, (this also affects my other, non-blog site to a lesser degree) comments and trackbacks are disabled and the files for these are either renamed or deleted.

I was searching for a way to block this spam, and the most common approach was using htaccess. I put together a piece that looks like this:

RewriteEngine On #only include this line once to enable the rewriting engine

deny from 84.174.72.102
deny from 217.185.183.70
deny from 205.234.145.222

RewriteCond %{HTTP_REFERER} ^http://(www.)?.*(-|.)?debt(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.*(-|.)?consolidation(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.*(-|.)?equity(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.*(-|.)?poker(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.*(-|.)?casino(-|.).*$ [OR]

(etc ... snip)

RewriteCond %{HTTP_REFERER} ^http://(www.)?slamhost.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?spielepsychatrie.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?superface.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?texasholdem.*$ [OR]

(etc ... )

RewriteRule .* - [F,L]

Originally I tried including a write-to-log piece I picked up during my searching,

 # Set an environment variable "BAD_GUY" so we can send their logs to a different file
# Be sure to leave out OR on the last RewriteCond, or your RewriteRule will -always- be
# executed in some versions of Apache
RewriteRule ^(.*) %{HTTP_REFERER} [R=301,E=BAD_GUY:1,L]
# Because we have set E=BAD_GUY above, we can do this in our log file:
CustomLog /var/log/apache/access.log combined env=!BAD_GUY
CustomLog /var/log/apache/access_bad.log combined env=BAD_GUY

but this gave me a '500 internal server error' (when viewing supernaut.info) when the .htaccess (permissions set to 644 rw-r--r--) file was placed in the supernaut.info folder, (as well as I'm not sure of the correct file path for CustomLog). Without that last piece everything seems to run ok, but I don't really know how to check or if it will even make a difference. Any ideas on how to improve this, confirm it's blocking, and include the logging to a separate file.

The second part is relating to hotlinking, which I get a bit of.

Currently I'm using this:

RewriteEngine  on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^https?://(www.)?supernaut.info/?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https?://(www.)?francesdath.info/.*$ [NC]
RewriteRule .(gif|jpg|jpeg|png|mp3|mpg|avi|mov|pdf|swf|tif|tiff)$ - [F,NC]

My images are arranged by year-month folders in the images/ folder, eg: /images/06apr/foo.jpg. The main question here is that from my experience (and htaccess stuff is not really something I spend much time contemplating), this .htaccess file needs to be in the folder with the image files to work, i.e. in images/06apr/ rather than the next level up, images/ which only contains folders. Is this correct?

Thanks heaps for any help


Frances

Tags: internal server errordenyapachespamhttpaccess logappreciatehtaccessenvironment variablemovable type