Herself’s Webtools

Ever wonder about them? I did. Years ago I noticed all sorts of urls showing up in my access-log files. Along with them were some strange referrals from sites that had nothing to do with my site’s subject matter. Most of the referrals were from porn sites. I’d go back to the site to see why they were linking to my site and there’d be no link there. Things got quiet but I’ve noticed them starting to show back up again.

What is happening is that access-logs are sometimes not protected and therefore viewable by the public and by search engines. A link is a link is a link so the less reputable sites would stuff your log files with links to themselves so as to improve their search engine ratings. Having your site associated with porn sites does not do you any favors, unless of course you are running a porn site as well.

How is this done? Bots are sent out with user agent strings that are links to the porn sites. 1×1 pixel images are linked to your site so every time a page load happens your site is referenced from them. Scrapers load up every page on your site, leaving a link in your access file to the porn site for each file fetched. Also browsers are hijacked and every time a hijacked browser visits you a link is left in the access file.

The best way to discourage this is to make sure your access-logs are not public and that you have blocked search engines from crawling them, see Robots.

If you have a persistent problem with specific sites use your .htaccess file to block them or send them to somewhere more appropriate say, whitehouse.gov.