I have NewRetirement.com set up to send me an email any time the site throws an error (including 404's). Its a great way to ensure that users receive a quality experience with the site, but its also quite informative. For example, you learn quite a bit about search engines by watching the behavior of their spiders. Yahoo! has been notoriosly slow about getting changes propagated through their system. We've moved some content around almost a year ago and set up 301 (permanent) redirects. While the Yahoo! slurp spider comes over frequently, on occasion it still requests old URL's.
What prompted me to write this post though was a new (to us) behavior of Yahoo! slurp. Today it threw a dozen or so requests at us for completely bogus URL's, all having this string in common: “SlurpConfirm404“. To Yahoo!'s credit, however, they do provide an explanation here:
http://help.yahoo.com/help/us/ysearch/slurp/slurp-10.html