Live.com Spambot Ignores robots.txt

Oh, MSNbot, when will you ever learn? I won’t rehash the story that lead me to blocking MSN’s referral-spamming bot, and that seems to have worked a bit. The problem is that the referral spam is still coming in! Yes, MSNbot is blocked but the spammy hits are still coming in.

Case in point, this hit from today over at Social Alerter:

/tips/how-not-get-dugg
  • At: 19 April 2008 11:04:39 AM GMT
  • Referred from: http://search.live.com/results.aspx?q=alerts&mrt=en-us&FORM=LIVSOP
  • Remote: livebot-65-55-165-107.search.live.com (65.55.165.107)
  • Request: HTTP/1.0 GET
  • Accepting:
    • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
    • Charset:
    • Enconding:
    • Languages: en-us
  • UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
  • Cookies:

Is it just me or is this beyond comical now?

Posted in Web programming | 2 Comments »

Subscribe to Things of Sorts

If you liked this post, please subscribe to the Things of Sorts RSS feed:

Leave a Reply