Having visitors on your website is awesome. However, you surely don’t want visitors who are trying to hack your website or who leave spam comments. I’m showing you how to identify and block those users by IP, user agent or referrer.
Why would you block some users from visiting your website?
First, I have to tell you that you are welcome on my website 😛
However, nearly every day there is someone trying to log into my admin account by using common passwords. Others steal my time by submitting shitty comments everywhere. I surely don’t want them to come back, so they’ll get locked out forever.
I know that often the person will be using proxies or it will be a bot automatically performing these actions. However, I also noticed that it’s often the exact same IP coming back or a specific IP range causing trouble. This motivates me to do the work of locking them out.
Some of my small traffic websites have another problem, that is, they are getting scanned by different web tools. Those will appear in Google Analytics with a 100% bounce rate and falsify my statistics. If a site has 40 visitors daily and 20 are coming from these tools, I’ll have problems finding information about the 20 human visitors.
One of these annoying tools is semalt, which seems to be some keyword monitoring service. It always instantly bounces of a page. I did not invite them to monitor anything on my website, they use my resources and destroy my statistics. That’s enough for me to lock them out.
How to block unwanted users
The best way to block unwanted users from visiting your website is by using a .htaccess file. .htaccess is a configuration file for servers that allows us to set up specific rules for a directory. In this case, we want to lock out users either by IP, user agent or referrer.
I recommend the following steps to advanced users only! Please ask your web hoster for advice on how to block unwanted users if you have no prior experience with web servers.
Log in cPanel or whatever else you use to manage your server. Navigate to the directory of your website.
Chances are high that there already is a .htaccess file. In this case, first try to open it. In cPanel, the option is called “Code Edit”.
If this option doesn’t exist, download the file to your computer and open it in a text editor like TextPad. Write the code I’m about to show you at the bottom of the existing .htaccess file.
If there is no .htaccess file, first check if there is an option that has hidden those files from your view. If this is not the case, create a new .txt file on your desktop. After writing the code I’m about to show you in the .txt document, you’ll be uploading it on your server and then changing the file name from blabla.txt to .htaccess. Please note that there is no text in front of “.htaccess”.
The exact code also depends on your Apache version. My web hoster uses Apache 2.2, so the code might causes trouble on another version.
Block users by IP
You can use plugins or tools such as iThemes Security to find IPs causing trouble, e.g. strangers trying to log in your admin account. Those tools will show you the exact IP address of the troublemaker you can then block automatically through iThemes Security or manually in your .htaccess file.
The code for blocking users by IP looks like this:
order deny, allow deny from 172.45.6.7 allow from all
What we are basically saying here is: „check if the visitor has the IP 172.45.6.7, deny access if it’s this IP, allow access for everyone else”.
We can also deny access for whole IP ranges. To do so, we just have to shorten the relevant part of the IP. For example,
deny from 172.45.6
would block all IPs from 172.45.6.1 to 172.45.6.255
Block users by user agent
With blocking user agents we can hinder some known bots from visiting our website. Those bots are mostly used to scan a website for security issues. The findings then get used to hack the website.
If you are using WordPress, install the plugin iThemes Security. It already has a built in list of bad user agents and you don’t need to do anything yourself.
Everyone not using WordPress can use lists of bad user agents from perishablepress.com or hackrepair.com and block them manually.
The code to lock out a bad user agent looks like this:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^BadAgent [NC]
RewriteRule ^.* - [F]
What we are basically saying here is “check for the user agent BadAgent and return a 403 forbidden status to it. We also don’t care if the letters of BadAgent are written lowercase, with capital letters or mixed.”
If you want to block several user agents, you have to add “OR” to the final brackets in the line where you state the user agent. This means we want to deny access when either one of the conditions (bad user agent 1 or bad user agent 2) is met. For example,
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^BadAgent [NC, OR]
RewriteCond %{HTTP_USER_AGENT} ^BadBoy [NC]
RewriteRule ^.* - [F]
For three or more bad user agents, we have to write OR at every line containing the user agents except for the last one:
RewriteCond %{HTTP_USER_AGENT} ^BadAgent [NC, OR]
RewriteCond %{HTTP_USER_AGENT} ^BadBoy [NC, OR]
RewriteCond %{HTTP_USER_AGENT} ^EvilAgent [NC]
Block users/sites by referrer
The referrer shows from which website people come on your site. We can block all visitors coming from a specific site. This makes sense when a bunch of spammers or hackers come from a specific site, for example, because your website has been posted on a shady forum.
You can find the referrer of your visitors through using tools like Google Analytics.
The code to block by referrer is very similar to the one for blocking user agents:
RewriteEngine On
RewriteCond %{HTTP_REFERER} example\.com [NC]
RewriteRule .* - [F]
In this example, if the referrer is “example.com”, we return a 403 forbidden status. It also doesn’t matter if the letters are lowercase, with capital letters or mixed.
This is the code I need to block the annoying semalt tool I just told you about.
RewriteEngine On
RewriteCond %{HTTP_REFERER} semalt\.com [NC]
RewriteRule .* - [F]
If you want to block several referrers, the same rules as explained under user agent apply (add “OR”).
Block whole countries
You can also block whole countries. Please keep in mind that there are options in Google Webmaster Tools that allow you to set up which countries your website aims at. Locking out whole countries is a very drastic method!
It can make sense in a few cases, e.g. when you provide a service that only works for people in the US, you write that everywhere on the page, but you still have a lot of people from other countries signing up for your service.
Blocking whole countries works through blocking IP ranges. If you are on WordPress, get the plugin iQ Block Country .
Otherwise, you can use this online tool to select the countries and get the .htaccess code that blocks the typical IP ranges of the country.
Did you have any negative experiences with your users? Let me know in the comments below.
