Every single visitor and bot that visits your site uses your precious server resources to serve up pages.
That’s great when they are REAL visitors with GOOD intentions – or real GOOGLEBOT – but, unfortunately, nefarious visitors can soak up resources too.
There’re a lot of different ways to tackle problematic bots, but today I’m going to show you how to get rid of people/bots that are hitting on suspicious URLs.
Usually, if a person/bot is repeatedly hitting on a suspicious URL – or multiple 404s over and over – it’s because they are LOOKING for a website vulnerability.
You need to make them go away before they find one and hack you!
First I’m going to show you two ways you can identify these nefarious visitors by their IP address and then I’ll show you what to DO with that IP address once you know it.
Using IP Deny Manager To Get Rid Of Dangerous Visitors
1. Enable The Redirection Plugin To View 404s
The Redirection Plugin for WordPress is a fantastic tool that you may already be using. If you’re not using it, go install it, and use it to monitor for 404 errors for a few days to get this information for you.
I configure the redirection plugin to keep a week’s worth of redirection and 404 data. Be sure you don’t let it keep more than that (but you can keep less than that) or you might end up with massive log files cluttering up your database.
Once you have a few day’s worth of logged data, go into the plugin’s settings page, and then into the “404s” link at the top of the page, and view all the 404s.
From here, you will often spot patterns of people abusing the system or looking for “doorways” into your site. Once you spot that, hover your mouse over their IP address in the right-hand column and then click the “show only this IP.”
In the image above, you see that an individual was looking for login pages. Obviously, he doesn’t BELONG looking for random login pages and needs to go away… so I copy his IP address down, to use in step #3.
You will find people looking for logins, looking for vulnerable plugins that don’t exist, looking for vulnerable URLs from known exploits, and lots of different stuff. It’s all about being able to see them be sneaky. (Consult with your web developer – either me or your own – regarding any you feel uncertain about.)
2. Temporarily Enable Live Traffic View In WordFence To Catch Multiple 404 Abusers
The WordFence Plugin for WordPress is one of my favorite security plugins both for its Live Traffic feature and because it’s one of the only tools available that can handle bot throttling. It also scans from the “inside” rather than only from the “outside”, which I feel is a much more valuable scan.
(Please note, you do NOT want to leave Live View running all the time because it uses a ton of server resources, and our goal is to reduce resource use. Toggle it on, get your info, and toggle it off. )
If you just installed WordFence you may not have much data for a day or two (no need to run live view during that time), but after a few days, turn on Live View and flip to the “Top 404s” tab and have a look at who’s wasting resources for you!
I generally do NOT remove IP addresses with less than 20 hits from THIS page. (I do remove a lot of smaller ones – who hit suspicious URLs – using the method in step #1.)
So once you see who your culprits are, jot down their IP address for use in step #3.
I will note that because the bulk of my traffic is from the USA and because IP addresses are SHARED resources, and I want to avoid blocking real people, I generally avoid blocking a USA address unless it’s wildly out of control or hitting known dangerous-to-me URLs.
3. Log Into Your cPanel and Use “IP Deny Manager” To Block These IP Addresses
Now we’re going to log into our hosting’s cPanel and use the IP Deny Manager (located in the “security” grouping most of the time) to let the SERVER handle these bad boys.
We’re letting the SERVER do this – rather than WordPress – because the server is designed to do this with a LOT less resource use than it takes for WordPress to do this. Using the server method prevents these bad guys from even getting a single visit again. They get a cPanel error page and are kept safely away from our sites, with very minimal increase in server resource use by us.
Now take your list of IP addresses and Add them in here one IP address at a time. Put them in that box (making sure there are no extra spaces before or after the numbers) and click add.
It’s easy to see what IP addresses are currently blocked and – should you ever need to unblock one – you can easily click remove.
Not only does this let the server do the heavy lifting – but if you ever change security plugins or remove WordFence – these guys stay blocked! (As long as you don’t migrate your site to a web hosting company that does not offer cPanel.)
How often you should go through these steps is really up to you. Older sites/installations often have more attempted abuse (or abusive crawling) than newer sites. However, newer sites are more likely to have unclosed vulnerability holes in many cases. You DEFINITELY want to do this regularly if you’re starting to exceed your hosting resources.
That’s all folks! Say bye bye bad guys!
~ Kim ~
Smart Tech For Smart Marketers
P.S.: Want to learn more about using your cPanel to improve your website and protect your business? Be sure to check out the cPanel training I have available for you here.