Stop Competitors from Stalking Your Website Using AdWords
Regular readers will know that I like to gaze at my log files in search of life-changing inspirational moments. Well I have another such gem of an inspiration for you: figuring out if someone is stalking your website using the Google AdWords keyword tool and how to stop them.
When someone goes to the AdWords keyword tool and asks for keywords based on the contents of a web page (the "Website content" option), Google actually requests the page live. This request shows up in the logs and can of course be blocked. The details are:
So what to do? Be careful blocking the IP addresses as a general precaution against stopping legitimate requests from Google IP addresses (Googlebot, Google's Feedfetcher, etc). However, the user agent is a good tell-tale sign and is ripe for blocking.
So: aim... fire!
Fire what though? A simple block? Nah, not much fun that. Knowing full well that only competitors will use that service to check out which keywords your pages might rank for, I would feed the requests dud content. Lorem ipsum anyone? How about random content about keyword theft? Here is an SEO exercise for you: which keywords can you get the Adwords keyword tool to show about your pages? To rephrase: what keywords can you "rank" for in the tool?
And don't forget to go back into your logs and see how many times people have stalked you.
Subscribe to Things of Sorts
If you liked this post, please subscribe to the Things of Sorts RSS feed: 
July 8th, 2008 at 2:07 pm
Hm. One thing though… what if Google uses the information obtained from that bot? They (they being Matt Cutts) already talked about how they make the AdSense bot do double duty that way, I could easily see them doing the same as a general rule for all of their specialized bots.
If that turned out to be the case, wouldn’t you be shooting yourself in the foot?
Or, worse yet, since it has a different User-Agent and comes from a different IP block than the standard Googlebot, what if they used it as one method of looking for cloaking issues? Not saying that they are or would, just that it’s conceivable.
July 8th, 2008 at 2:22 pm
All good points Michael, but that’s no fun
Sarcasm aside, I think it’s prudent to do a full analysis of the IP addresses Google uses to see if there is overlap in IP addresses between different user agents/services. That would be good evidence for your ideas.
P
July 9th, 2008 at 3:51 am
Good ideas, both of you