Posted by ljmacphee on November 7, 2007 under security, tools, wordpress | 
WP Plugin to prevent bot registrations, the professors tool is good and I still recommend it, I just wrote one I think is a bit better.
Ban email domains from user registration plugin
Writing a plugin to ban email domains from user registration was getting close to the top of my todo list but someone has already done it. ( Cool for me, one less thing on the todo list. ) I had been deleting all users who registered but didn’t comment every morning over coffee.
I have to say I’m baffled. I’m not sure what spammers gain by registering as a user on your blog but not posting any comments? My guess is the comments are coming later. Or there is some new trick we haven’t yet figured out about to appear.
Posted by ljmacphee on October 29, 2007 under security, tools |
** update: I wrote a WP plugin to block most bots WP Security Plugin if you are having trouble with bot registrations try WP plugin bot blocker
I’ve one site that seems to attracts bots. It was attracting bots when it was just a directory on TimesToCome eight years ago. Who’d've thought house plants would be such an attraction for evil bots? There are several sites with directions for building bot traps. The main idea is the same for all.
First you create a bot_trap directory. Second you add a line to your robots.txt file telling legitimate bots not to enter that directory.
User-agent: *
Disallow: /bot_trap/
Then you put a hidden link in one of your busy pages, perhaps the main page or archives page that humans can’t see but that bots can see. <a href=”/bot_trap/got_you.php”><img src=”/images/1_pixel_clear.gif” border=”0″></a>. Since most evil bots are aware of the clear image you should give your 1×1 pixel image a different name.
Lastly you create a got_you.php file and place it in your bot_trap directory. What to put in your got_you.php file is a matter of taste. You can have the got_you.php file, email you, you can ban the ip address by adding it to your .htaccess file, or you can do evil deeds to the bots ip address.
PHP email and ban with .htaccess examples
Kloth: Bot trap emails you and bans ip number
735: Ban bad bots and email yourself
Or you can just write the bans into your .htaccess file yourself.
A close to perfect .htaccess ban list
Block known evil bots by manually re-writing .htaccess
An alternative for those of you not comfortable messing with .htaccess is to add a page ( The ultimate spam bot bait and trap page ) to your site with a hidden, banned link.
You do need to be careful. The smallest mistake in .htaccess will take down your site. Also you don’t want .htaccess to get so large it slows down your website. Another concern is that you don’t want to trap and ban good bots. Banishing Googlebots, sending Yahoo bots to the depths of the internet will not do your search engine rankings any favors.
Since my bot attacks are not coming from known bots I’ll be using one of the php trap and ban versions on my sites.
More information:
Everything you wanted to know about bots
Posted by ljmacphee on October 26, 2007 under security, tools |
There are good bots and bad bots. Some bots crawl your site and stick you in their search engines. The Google bot is your friend. Some bots scrape your site for email addresses, or just to copy your site. Bots are small programs that traverse the web, usually traveling from one link to another and downloading part or all of what they find.
You can tell by looking at your log files when you’ve been botted. Several pages will have been loaded in a very short time by one ip address. Often the pages will be loaded in alphabetical order, or by the link list you provide to various pages.
So if you see a bot has been viewing your website how do you know who it is?
BotSpot: The List of all bots
Kloth.net Bad Bots List
Robotstxt.org, Database of Web Robots
IP Addresses of Search Engine Spiders
Search Engine Robots
List of User-Agents ( Spiders, Robots, Browsers )
What can you do about bad bots? Probably not much. Some hosting services let you ban specific ip numbers from getting to your site. However, bots don’t always come from the same ip number twice.
There is Bot Trap ( I haven’t tried it but is sounds promising)
Fleiner has some tips on how to ban bad bots using your .htaccess file. There are also some bot traps available for download on that site.
** update: I wrote a WP plugin to block most bots WP Security Plugin if you are having trouble with bot registrations try WP plugin bot blocker
Posted by ljmacphee on September 24, 2007 under tools, useful sites |
Google Webmaster Tools are useful tools for your websites. Webmaster tools give different information than you will get from Google Analytics which gives visitor information.
Webmaster tools will tell you when you have broken links and what queries people use to find your site. The statistics page will also show your websites position in the search results for specific queries.
You will find the terms people use when they link to your site and which pages get the links. Google Webmaster Tools also tells you your highest ranking page by month for each site.
Even though you likely already use analytics, this tool too has its uses. Like all the Google webtools it is free.
If you are not already reading it be sure to check out Google’s Webmaster Blog. Which is full of useful information for webmasters.