Welcome Reddit, Digg, and StumbleUpon users! If you like this post, please vote below. Thank you!

Update 2: Yuri explains more background and asks What happens next?. Reuben Yau and Kichus have both blocked the IP addresses. Boy are people angry.

Update 1: DazzlinDonna from SEO Scoop has written an excellent background to this fiasco, and Michael VanDeMar is reporting that Microsoft is interfering with AdSense. Ouch.

The bot analysis continues, and this post presents evidence indicating that Microsoft is spamming websites. A big claim, I know, but I can't find a better explanation. You'll have to decide.

The summary: IP addresses belonging to Microsoft are requesting pages from eKstreme.com and blogSci.com (my science blog) with HTTP referer headers suggesting that the hits were from live.com searches. These referer headers are spoofed as the keywords from these supposed searches are sometimes in no way related to the requested page. Additionally, for most of the other supposed searches, the requested pages do not rank in the top 10 (first page of results) in a way to send this traffic.

For some odd reason, the webmaster community has known about this for a couple of months. In September, SE Roundtable posted about other webmasters complaining about this spam. Surprisingly, we also got official confirmation (via a WMW thread) from msndude that this indeed happening and it's (and I'm quoting) "part of a quality check we run on selected pages". This is an unacceptable explanation as you'll see from the data below because it has none of the hallmarks of a quality check but all the marks of referral spam.

The hits discussed below are extracted from the blogSci.com data to keep things simple, but a similar data set exists for eKstreme.com.

The Hits

The whole list of hits is way too long to quote in full here, so here is a sampling of my favorite requests:

/randoms/academic-publishers-as-spammers

• At: 17 August 2007 05:53:27 PM GMT
• Routed to: /index.php
• Referred from: http://search.live.com/result.aspx?q=make+money+online&mrt=en-us&FORM=LVSP
• Remote: bl2sch1082213.phx.gbl [] (65.55.165.119)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  • Charset:
  • Enconding:
  • Languages: en-us
• UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
• Cookies:

---

/science/water-vapor-found-outside-earth

• At: 18 August 2007 03:05:43 PM GMT
• Routed to: /index.php
• Referred from: http://search.live.com/result.aspx?q=make+money+online&mrt=en-us&FORM=LVSP
• Remote: bl2sch1082008.phx.gbl [] (65.55.165.66)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  • Charset:
  • Enconding:
  • Languages: en-us
• UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
• Cookies:

---

These two hits above are the first I have in my records. What's amusing about them is that both supposedly came from a search for [make money online].

/science/magnets-make-rainbows

• At: 19 August 2007 03:55:48 AM GMT
• Routed to: /index.php
• Referred from: http://search.live.com/result.aspx?q=ticket&mrt=en-us&FORM=LVSP
• Remote: bl2sch1081815.phx.gbl [] (65.55.165.25)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  • Charset:
  • Enconding:
  • Languages: en-us
• UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
• Cookies:

This one is also very random: a blog post about a cool new magnet-based technology to create colors is ranking in the top 10 for the query [ticket]? Not even Live.com generates such irrelevant results.

Anything more recent? Sure:

/science/osteoporosis-genes

• At: 11 November 2007 03:26:43 PM GMT
• Routed to: /index.php
• Referred from: http://search.live.com/results.aspx?q=osteoporosis&mrt=en-us&FORM=LIVSOP
• Remote: bl2sch1081815.phx.gbl [] (65.55.165.25)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  • Charset:
  • Enconding:
  • Languages: en-us
• UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
• Cookies:

---

/technology/filling-amazons-orders

• At: 11 November 2007 03:29:24 PM GMT
• Routed to: /index.php
• Referred from: http://search.live.com/results.aspx?q=amazon&mrt=en-us&FORM=LIVSOP
• Remote: bl2sch1081909.phx.gbl [] (65.55.165.43)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  • Charset:
  • Enconding:
  • Languages: en-us
• UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
• Cookies:

At the time of writing, there are 245 such hits in my records since August 2007.

Not convinced? There is more. Some of these hits came within seconds of being indexed by MSNBot. The pattern is like this: the page is requested by MSNBot (which is authenticated, so it's genuine) and within a few seconds, the very same page is requested as described above with a live.com search are referer. An example:

/science/computer-solves-problem-without-running

• At: 10 November 2007 12:05:14 PM GMT
• Routed to: /index.php
• Referred from: (No referer.)
• Remote: livebot-65-55-209-143.search.live.com [] (65.55.209.143)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: text/html, text/plain, text/xml, application/*, Model/vnd.dwf, drawing/x-dwf
  • Charset:
  • Enconding: identity;q=1.0
  • Languages:
• UA: msnbot/1.0 (+http://search.msn.com/msnbot.htm)
• Cookies:

/science/computer-solves-problem-without-running

• At: 10 November 2007 12:05:36 PM GMT
• Routed to: /index.php
• Referred from: http://search.live.com/results.aspx?q=problem&mrt=en-us&FORM=LIVSOP
• Remote: bl2sch1081810.phx.gbl [] (65.55.165.20)
• Request: HTTP/1.0 GET
• Accepting:
  • HTTP: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
  • Charset:
  • Enconding:
  • Languages: en-us
• UA: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
• Cookies:

The typical delay between the indexing request and the spoofed search hit request is 5-20 seconds.

How to Recognize the Fake Hits

Anyone staring at these hits long enough will see some signatures to detect them:

• Note how all of them have identical user agents (UA field) and pretty much everything else is identical (bar the the requested page and the referer).
• The IP adresses all belong to the same C-block, namely 65.55.165.*.
• All of the query strings in the live.com referrers have &mrt=en-us in them. Here in the UK, I get &mkt=en-gb when I really use Live.com for a search.

Needless to say, this smells like bot behavior.

An Analysis

Let's think about this for a minute: What on Earth is going? Why are these hits happening? I can think of two explanations:

• The tinfoil/sinister explanation: pure spam from MS. Why? So that webmasters see Live.com referrals coming in increasing numbers. This is not hard to hide: if you only get like 10 referrals from live.com a month, another 10 is a doubling but which sad webmaster would check those out (apart from me)?
• The "surely not" explanation: this is an automated way to check the search results to see where pages rank for keywords the page could potentially rank for. This is what msndude confirmed in the WMW thread, but as you can see above, it doesn't really look like a quality check. Also, if this is indeed a quality check, why not run it on the cached pages and not alert (and annoy) the webmasters? Microsoft have full access to their index and they should use it!

I subscribe firmly to the first explanation: the search keywords are spammy in some cases, always too general, the requested pages never rank in the top 10 as the referring URLs would suggest, the hits have identical user agents (i.e. not the typical variation you would expect from various people using normal browsers on different operating systems withing the same company to show) and the actual referring URL does not match what a human being searching on live.com generates.

In short: it's spam and not a quality control check. What do you think?

This is a paid review.

The first time I heard about SponsoredReviews.com was back in January. It was described as a service not very different than ReviewMe (reviewed here) and others. So now that SponsoredReviews has launched, I thought I'd take a look...

The first thing to note is that it is quite similar to the other blog review services. You log in, add your blog(s) and wait for advertisers to find you. However, there is a lot more here: firstly, you can set a desired price to charge for each review, something new to me. SponsoredReviews calculates a default price, but you can over-ride that. I did (tripled it) and I still got an advertiser to request a review! Not a bad move on their part to allow for these things. In essence, SponsoredReviews is actually a free market of sorts in that the sellers of blog space (the bloggers like me) punt their services to the buyers (the advertisers). If the price is too high or too low, it should naturally find its stable equilibrium. Incidentally, SponsoredReviews.com pays you 65% of whatever you charge, a bit like a tax for using their system. It's a very insightful move on their part.

But enough market economics, what else is there? The second cool feature is that bloggers can go out and find advertisers. I have mixed feelings about this as I see a conflict of interest: if you approach an advertiser offering a review, chances are you already have something good to say. If an advertiser approaches you out of the blue, I believe there would be more objectivity. Either way, time will tell how this plays out, and no doubt I'll approach an advertiser or two just to get a feel for this.

That's all good, but I'd like to end with a little note to SponsoredReviews: tone down your language a bit, willya? I'm referring to an email (I presume automated) I received today. SponsoredReviews gives bloggers seven days to complete a review once both parties accept it. Today is the 7th day for this review, and so I received an email reminding me. The wording rubbed me off the wrong way:

You are receiving this email because you have reviews that must be finished today in order for them not to be canceled and money refunded back to the advertiser.

Since we give our bloggers more than enough time to finish a review (7 Days) we take unfinished or late reviews very seriously. One of the things we will enforce is our right to suspend accounts to those bloggers who are not willing to comply with our guidelines.

Threatening to suspend accounts is bad enough, but it's made worse by the fact that you cannot cancel reviews once you accepted them (not that I can see anyway). I don't know about you, but sometimes I get caught up in other important things (like eKstreme.com being deindexed by Google; more soon), so the SponsoredReviews system is kind of a trap. Their email states that you can contact them to request more time, but they are very hand-wavy about how they may let you off. Gee, thanks.

Besides, if the review deadline is so darn important, why not send a reminder 3 days after acceptance or a day before the due date? Why leave the threat of termination till the very last second? That doesn't make sense.

All in all, SponsoredReviews is a neat system, that I think trumps ReviewMe in some respects, but also is worse in others. The good thing for us bloggers about all this activity is that the competition will (should) make the paid reviews systems better and the payouts should get better. The fun is about to being.

Technorati Tags: SponsoredReviews, sponsored reviews

The giant website Answers.com is adding a 'blogs' section to their pages, and you can submit your blog here. All submissions are reviewed by editors (can you imagine the work load?) so it might take a while. They don't list any criteria for inclusion, apart from asking 'Is there a blog that you think other Answers.com users would be interested in?'. I sure hope so, as I just submitted this blog

Not sure how new this is, but it was certainly news to me.

Technorati Tags: answers.com, blogging

By now you must have heard of ReviewMe. If not, here it is in a nutshell: it's an advertising website that pays bloggers to write reviews for products or services from advertisers. The only (serious) condition is that bloggers have to label such reviews as paid-for, and so to comply, I hereby label this review of ReviewMe as sponsored and paid for.

Objectivity?

This raises the issue of objectivity: can bloggers (or any reporter) be truly objective about what they are reviewing if they are getting paid for it? I would answer that, in this particular case, yes, objectivity can be maintained. You see, the set up is that bloggers have a relationship with ReviewMe, not with the advertisers. Further, since the advertisers ask to be reviewed by certain blogs, the bloggers can choose not review any or all products. Likewise, advertisers can choose who they would like to review them. If you think about it, this is the same set up as any magazine, journal, or newspaper, such that the bloggers are freelance writers for the publication, which in turn attracts advertisers. The only difference is that the published articles are not on paper or on a website, but distributed across many websites.

This system will mirror what we already have: some will write glowing reviews for everything, attracting more advertisers but losing their visitors' trust, while others will be completely negative, attracting no advertisers, with the great majority maintaining some degree of objectivity. I hope you'll believe me when I say I will strive to be in the lattermost group!

How does it work?

Create an account and login to your control panel (screenshot below). The interface is clean with clear blocks of information telling you of any messages and a summary of your reviewing activity. The account allows you to be a blogger writing reviews and be an advertisers if you want reviews.

Through your account, you can submit up to six blogs to be included in the approved list of reviewing websites. Reviews start with advertisers requesting reviews blogs they choose from the ReviewMe directory (for example, my things of sorts entry and the Blog of Science entry). The blogger can then accept or reject the request.

Will it work?

I don't see why not, but the objectivity question has been raised many times around the web. A wrong slip might tarnish the ReviewMe image and the system beyond repair, so the onus really is on the ReviewMe team to be fair - and strict - umpires to manage everyone. From their FAQ for advertisers as I write this:

Can I require a positive review?

We do not allow advertisers to require a positive review. The vast majority of reviews are measuredly positive, although many do contain constructive criticism. We view this as a bonus: how else can you quickly and cheaply get feedback on a product or service from influencers?

IMHO, they need to be much more strict and rigid in enforcing a fair framework about what to expect from reviews and reviewers - Criticism taken the wrong way will start fights.

The other issue is getting enough advertisers to request reviews. Inventory-wise, a large selection of blogs with little or no reviewing history might put off advertisers. If you know that a given blogger in your target niche is a good reviewer, why go elsewhere and risk losing money?

All in all, I think ReviewMe is a good idea and it will succeed. Their launch promotion is certainly a great way to kick off (talk about the SEO benefit!), but let's see what they deliver in the next few months. With this in mind, expect a few more reviews here on things of sorts or on the Blog of Science.

Technorati Tags: ReviewMe, reviews, blogs, blogging

Dr. Nora Ganim Barnes at the Center for Marketing Research at the University of Massachusetts Dartmouth surveyed 74 top bloggers about their experiences. The 60-page paper (PDF) is a tome of wonderful advice for starting and running a blog. After a commentary about the results, explaining the major themes in the responses, the raw data of the 22 questions in the survey are presented. It's really a good read.

Via Micro Persuasion.

A while back, I replied to a Digital Point thread about marketing a new blog. My reply basically outlined the idea of 'blogs are conversations' and the basics of using this idea to get a community around your blog.

Now Amy Gahran talks about another complementary and overlapping marketing strategy, which she calls Strategic Commenting. The idea here is that you engage in conversations at blogs related to yours. Again, the general theme of 'blogs are conversations' comes up again, but now with a new slant on how to converse.

Any more ways to converse? I would love to hear about them in the comments below!

When installing WordPress for this blog, I stumbled across this problem: I couldn't get 301 redirection to forward the www.ekstreme.com version to the ekstreme.com version of the site if accessing the /thingsofsorts/ directory. After some playing, this is how to do it.

The default .htaccess that comes with WordPress is:

The simple modification is this:

Nice and simple. You can read more about 301 redirection and SEO in my 301 redirection tutorial; it even includes code examples!

Technorati Tags: 301 redirection, Apache, WordPress, blog

This blog, things of sorts is my second blog. I'm starting a third. They are all based on WordPress. But, where could I have gone to set up a free, remotely-hosted, blog? Well you have quite a few options:

• Blogger. The grandfather of blogging software, Blogger is probably the most famous blogging site on the net. I used it at some point, and I liked it. Doesn't have categories though, which is a pain.
• WordPress.com. This sets up a hosted WordPress blog for you, from the same folks that wrote WordPress. Not a bad place to start learning blogging on a real blogging platform.
• MSN Spaces. I haven't tried MSN Spaces, but it's MSN's offering.
• LiveJournal. I have friends addicted to LiveJournal. They must be doing something right to get such a following.
• Newsvine. Newsvine has a nice interface and it gives you a subdomain as your own.

By no means is this a comprehensive list, and it covers only some of the big ones. The point is simple: if you don't want to mess with installing blogging software on your server, it's easy to outsource it.

The other thing for you SEOers out there: all these free services are hosted on different C-class IP addresses. I don't know, but isn't this is a good way to get free, on-topic, one-way links to your main sites? Hmmm

Technorati Tags: Blogger, WordPress, blog, MSN, Spaces, LiveJournal, Newsvine, blogging