New Stealth Crawler from Yahoo!

For the past few months, I’ve been tracking a crawler from Yahoo! that does not identify itself on my science blog. The bot’s details are:

Notice a few interesting details: No user-agent string, the fact it provides an HTTP_REFERER header that’s the same page being requested, it comes from *.yahoo.com not the usual yahoo.net for Slurp, and the fact it says "image" and "srch" in the host.

The tracking is very low-level, a few hits a day with lots of one-hit-a-day visits.

What’s really interesting is how laser-targeted it is: it’s only requested the same two pages many times since May. The pages are the specific blog post linked to above plus the archive page that contains that post, so it’s likely something about that post that’s of interest to the bot. And yes, the post contains an image, and the image is the only one in the main content of the archive.

I’ll dig deeper when I have a chance. Please let me know in the comments below if you’re seeing something similar.