Herself’s Webtools

LOL obviously these researches don’t run their own websites.

University Park, Pa. – Web site policy makers who use robots.txt files as gatekeepers to specify what is open and what is off limits to Web crawlers have a bias that favors Google over other search engines, say Penn State researchers whose study of more than 7,500 Web sites revealed Google’s advantage.

That finding was surprising, said C. Lee Giles, the David Reese Professor of Information Sciences and Technology who led the research team which developed a new search engine—BotSeer—for the study. . . . [read more Study shows Google favored over other search engines]

Why is Google preferred? Google provides the easy to use AdSense program to help webmasters earn some money, Google has Webmaster tools and Analytics for webmasters, and Google, right now at least, is the best of the search engines. However much better we would like Google to be, it hands down trumps the others. Probably because Google is so webmaster friendly.  But most important of all; it is Google who sends us the most traffic from search engines.

See also:
BotSeer, a search engine providing information on robot.txt files across the internet