Herself's Webtools

Your WP Security plugin only catches bad bots if you tell it who the bad bots are and what requests to blog. This is my current list of bad requests. You can just copy and paste it into the request blacklist after your current list and hit the update button and it will add them to your list.

12/3/09
DOCUMENT_ROOT

12/30/2008 While you can turn off trackbacks in WP 2.7 settings it doesn’t stop spam comments coming through that way. If you do not allow trackbacks add
trackback
to your banished request list.

12/30/2008 Nathan ( site unknown ) send me a great list of requests to block. But I’m only adding a few of them here since many will catch users who haven’t whitelisted themselves. These ones are all bad requests and didn’t give any false positives on WP

cmd.exe
root.exe
shell.exe
_vti_bin
cltreq.asp

8/29/08

$_GET
(java|vb)
.gif?
.jpg?
.txt?
.xml?
</script>
<SCRIPT>
ASCII
CAST
com_jd-wp
CONCAT
DECLARE
DELETE
DOCUMENT_ROOT
formmail
includedir=
index.php?template=
INSERT
lwp-trivial
OPTIONS
passwd
PATH=
POST /xmlrpc.php
PROPFIND
register++++
SELECT
SERVER
sidebar.php?
UNION
UPDATE
word-tube-button.php?
wp-config
wp-table-button.php?
wp-trackback.php?
x-aaaaaaaaa

---

Notes:

I’m also checking Emerging Threats Rules downloads, it is a great site to learn a bit more about security. Go to emerging web_sql and scroll down to the WordPress section.

8/28/08
<SCRIPT>
</script>

8/11/08
CAST
DECLARE

Grumpy old man emailed to tell me a MySQL injection attack had been tried on his site using these terms that had not been caught by the security tool. I’m adding them to my list today and suggest you do as well. See SQL injection attack using DECLARE or New SQL Injection Attack Infecting Machines for more details.

7/16/08
Removed wp-login.php?redirect_to=http%3A%2F%2F it keeps tripping on WP 2.6

7/9/08 All of these were part of several attempted but failed hacks recently
com_jd-wp
index.php?template=
board.php?see=ftp

7/6/08 I added ‘PATH=’ to the list

New additions 6/18/08 ( still testing these )
passwd

New addition 6/17/08
$_GET

New addition 6/15/08
register+++

New addition 6/13/08:
.gif?
.jpg?

Now your WP Security plugin only catches bad bots if you tell it who the bad bots are. This is my current list. You can just copy and paste it into the banished agent list and hit the update button and it will add them to your list.

12/3/09

</script>
<SCRIPT>
ac-baida
AnotherBot
botpaidtoclick
Click Bot
cr4nk
curl
DA 5.3
DataCha0s
discobot
EBM-APPLE
EmailSearch
EmailSiphon
FAST ESP Document Retriever
Firefox 2.0
Ginxbot
GrubNG
gvfs
HTTrack
Incutio
Indy Library
Internet Ninja
Java
JetBrains
libcurl
libwww-perl
lwp-request
lwp-trivial
Macintosh; I; PPC
Microsoft Data Access
MJ12bot
Morfeus Fucking Scanner
Mozilla Firefox 5.0
Mozilla/4.0(compatible
Mozilla/4.08
Mozilla/4.61 (Macintosh
Mozilla/5/0(compatible
Mozilla/7.0
Mozilla/8
Mozilla/Firefox
Mp3Bot
MSIE6
NIPGCrawler
PEAR
PECL
PHPot
Provider Protocol Discover
PuxaRapido
PycURL
Security Kol
Site Sniper
Sogou
sun4m
Sunrise
syncrisis
topicblogs
User-Agent
W3CRobot
w:PACBHO60
WebDav
WebRipper
Wget
window.location
Winnie Poh
www.ranks.nl
X12R1
Web::Scraper
Xerka-bot
SkyGrid
Python-urllib

---

More information:

The ultimate htaccess file ( long list of bots to block )

Spiders and bots to block ( long list )

Top 10 Spam bots to block

Top web robots comment spammers

Harvester user agents

Spider identification

---

Notes:

12/3/08
I added
ac-baida

12/8/08

I added

Web::Scraper ( are you kidding? )

Xerka-bot ( lots of unhappy webmasters on forums about this bot )

SkyGrid ( gathers info for stock investors – nothing to see here )

Python-urllib ( was hitting feed every 15 minutes )

11/29/08

Added “Morfeus Fucking Scanner” to the blocked bot list. David Read was kind enough to alert us to it this morning

9/8/08

I caught a new scraper today. “FAST ESP Document Retriever”

8/28/08

Interestingly I had an attack by syncrisis.com who tried to run the script in the user agent section rather than as a request. So I’m adding <SCRIPT>, </script>, window.location, syncrisis to the user agent field.

8/4/08

I have a user who tells me Mozilla/4.08 is a legit phone browser.  You might not want that one on sites likely to visited by cell phones.

8/1/08

I added lots of bots today. Python-urllib, AnotherBot, Mozilla/9, Mozilla:, PuxaRapido, SiteSucker, newLISP, yourname were all added for not identifying themselves by url or email and not using robots.txt. bot@bot.bot, PHP/5 had no id and excessive hits, Test was banned for stupidity, Atomic_Email_Hunter, Jakarta, LeechGet, libwww-FM, WWW-Mechanize, and core-project were all banned for attempted badness.

7/23/08 I added fake browsers Mozilla/8 and Mozilla/Firefox to the list. I also added the W3CRobot. It is an open source webcrawler that can be used for good or evil. One of them hammered my personal website so I’m banning it. Do as you choose. Also I added topicblogs. Seems they have scraped lots of websites and all they say is coming soon. No way to tell if they are good guys or bad guys so I put them on the block list.

7/20/08 Lots of bad guys this week: ‘Indy Library’ appears to be an unidentified image grabber, sun4m, EBM-APPLE, both tried cross site script attacks, EmailSearch is an email scraper, NIPGCrawler and W3CRobot appear to be scrapers.

7/10/08 Bandwidth is down 1/3 on websites, number of human visitors is up. So the bad robots are getting filtered. I hadn’t realized how much bandwidth they took up. I found this bot trying cross scripting attacks

Macintosh; I; PPC

7/9/08 Lots of scrapers this week.

DA 5.3

Internet Ninja

7/6/08 Busy, busy little bots: I added 4 new ones to the list

Mp3Bot

gvfs

WebRipper

discobot

7/3/08

AVG is yet again hiding under fake user agents; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1), User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1), User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

If you have ‘User-Agent’ on your list as I do you will be blocking AVG Toolbar users who prescan websites. This is not AVG’s first run in with webmasters. Most of the older security programs blocked the last user agent which is also malformed. ( there is no space between 5.1; and 1813 ). I personally am leaving it blocked. All of you should make your own decisions. More information is here: AVG disguises fake traffic as IE6, also see How to beat AVG’s fake traffic spew.

Another concern is that ‘User-Agent’ in the user agent string is used by one of the top all time forum and blog spam bots. Unblock AVG and you unblock the spam bot. But that is why I left the bots in your hands. Block or unblock them as it suits you.

7/2/08 Winnie isn’t so cute, caught Winnie Poh trying to hack WP

Winnie Poh

7/1/08 Fake web browser

Mozilla Firefox 5.0

I’m also seeing several entries that have the bulk of the user agent zero’d out.

Mozilla/5.0 (000000000; 0; 00000 000 00 0 000000; 00000) 000000000000000000 0000000 0000 000000 0000000000000 0000000000000

So far I have not seen bad behavior from this user agent so I am undecided on whether or not to ban it.

6/25/08

I added in MJ12bot for hammering the site.

MJ12bot

6/25/08 Version 1.7 of the security plugin prevents the webserver from banning itself so be sure to block this user agent now.

Incutio

6/21/08

mozilla/5.0

Mozilla/4.61 (Macintosh

6/20/08

Mozilla/4.08

lwp-trivial

6/19/08

Blocking WordPress also blocks wp-cron so don’t use that one. There is also a website scraper that uses that user agent. So if you are not using cron jobs, block it, but keep an eye on it. I’ll try to find another way to block the scraper that uses that as a user-agent. You’ll know by the ip number whether it is you or a scraper being blocked.

You can banish WordPress/2.3, WordPress/2.5, WordPress/4.0 and any other versions other than the WP you are using.

Many webmasters ban ‘larbin’ and ‘Jakarta’ I have not yet had trouble with either, so I am not currently banning them.

6/18/08 New Additions: ( I am not blocking Firefox or IE these are fake user agents I’m still testing this list will add to main list if no problems tomorrow )

Internet Explorer

Firefox 2.0

Mozilla/4.0(compatible

Mozilla/5.0(compatible

WordPress

6/16/08 Many webmasters are having problems with AVGs out of control bot. Should you wish to block it, I am not, add the following bot to your block list:

Windows NT 5.1;1813

6/13/08 New Additions:

EmailSiphon

Microsoft Data Access

WebDAV

Click Bot

PHPot

lwp-trivial

Did you accidentally trap a Google Bot or Yahoo? I haven’t caught Google yet, but the Yahoo bot is not especially bright and sometimes gets stuck. First verify the ip numbers Robot ip numbers and be sure you caught the real thing, not a fake. Then just remove its ip number from the ip banished list.

Or do just lookup the ip number and see if it is used by who it claims to be.

I have xmlrpc.php in my robots.txt file as Disallowed. Both YahooSlurp and the Amazon zermelo ignored that and were flagged because they attempted to crawl that file. I just removed their ips from the ip list. In the future let’s hope they read the robots.txt file.