How passionate are your users?

There is a big fuss going on in the education world at the moment about a company called Blackboard acquiring its rival called Angel. There is a lot of commentary and mud-slinging going, so I'll leave it to you to dive into the controversy. In a nutshell, summarized by the Chronicle of Higher Education, the problem is that Blackboard produces (apparently) rubbish software and has horrible customer support. Angel (apparently), lives up to its name in that it provided access to source code, super-fast responsiveness and generally angelic customer support.

While this fiasco is being fought out, I'd like to take marketing lens to this. Blackboard and Angel held a meeting in Chicago to engage the uproar (+1 brownie points on this count), and I'd like to point out something subtle, yet very important, in this story. Quoting the Chronicle:

To show their preference for Angel and dislike for Blackboard, some wore blue and black ribbons — the colors of Angel's logo — to the Chicago meeting.

Tell me, how many of your users are willing to wear your colors? How many people are cheering for you just as they would cheer a sports team?

I find this as an awesome example of a company creating passionate users and a *community* around its product, and that to me speaks about just how good Angel's customer interactions are/were. It users are fighting for its independence. If your business was about to assimilated by another much larger and hated company, would your users care enough to fight or just move on? What can you about it now?

eKstreme.com Has Moved

If you're reading this, you're reading eKstreme.com on its new home. Pat your ISP on the back for having quick DNS changes :)

Why the move? Three reasons:

  • Costs.
  • Speed.
  • A good opportunity to upgrade the backend.

Yes the third reason is not, alone, grounds for moving servers, but combined in, it's a great chance to do some house cleaning.

I've also closed two tools:

  • The Backlink Social Celebrity tool as it was just too broken and not useful.
  • Keywords Extractor and Analyzer. This depended on the Yahoo! API which kept changing and it's not worth the time to keep it running. Maybe in the future I will have the chance to resurrect a better implementation.

If you miss either of those too much, let me know and I'll see what I can do.

Google Alerts Now Spell Checks the Queries

Lately I've been noticing a lot of weird hits coming in via my Google Alerts emails. I've dug into it and I think I've figured out what's going on: Google Alerts is spell checking the queries and matching the queries as it would do in a search. This in addition to matching the Alert queries exactly as previously. This new behavior kicked in about a week or 10 days ago.

For example: I keep an alert for [blogsci] because I have a website at blogsci.com. Up till recently, I used to get alerts only when the word "blogsci" was matched in a page. Now, I'm getting Alerts for pages that do not ever mention the word "blogsci" but the spell checked "blog sci". So I get matches for "...blog: sci-fi...". See what happened there?

Another example: I run a website with a domain name of XY.com where X is a word and Y is another word. My Alert is set to match it exactly as [XY]. This was going well until recently when I started getting alerts that match [X Y].

Another example: I have an alert for [cli.gs], my latest web app. I get a lot of spurious alerts for this because it matches [cli gs] which is a very popular combination apparently.

Anyone else seeing this weirdness? Any other interpretations? Thoughts in the comments please!

Bye Bye MyBlogLog

Hi MyBlogLog!

Part of what makes me so special is my ability to automatically add you to communities in which you have shown a repeated interest. I have just added you to the following communities:

1) Garbage Bin

http://go2/dev/null

Bonus -- The "Hot in My Garbage Bing" box on your My Home page will keep getting worse and worse with each community I delete my account from.

Rock on,

Me voting with my feet


Obviously, I'm pissed off about MyBlogLog, and anyone who has seen their emails similar to the above can guess why. The reason is quite simple: it's a mediocre service (at best) with a horrible interface and stupidly horrendous default behavior. As a webmaster, what I get out of it is rubbish analytics and an awkward way of building a community. As a user, I keep getting added (by default) to communities that I don't want to join necessarily. See, this assumption that by default I'd like to join any given community, wrapped in a self-gloating email ("Part of what makes MyBlogLog so special is our ability") is just plain wrong. I never turned on that feature and I never wanted to join that feature.

On the flip side, If I choose to manually join a community that means a lot more to the community's webmaster than me being added automatically to beef up their community head count. This works for the communities I run too: I don't want to just rack up avatar but I want a genuine community. Granted it's slower to build a community like that, but it will be by the members' choices not some automated "special" "ability". Please, get over yourselves.

So without futher wasting further time fixing stupid default behavior in return for little, I've closed my MyBlogLog account.

Hey YouTube: UK = GB, and both are English

Sometimes I see help messages that just leave me speechless. This message from YouTube about my automatically-set language preferences goes above and beyond anything I've seen in a long time because it has two big "WTF moments":

YouTube language preferences message

The problems?

  • The red circles: The suggestion that English (UK) is different from English (GB). Psst. They're the same thing. It's an exceptional reservation in the ISO standard.
  • The black circle: The whole message is apparently not in English because the link at the bottom right corner gives me the option to view it in English. When I click it, I get the same message, but instead of suggesting English (UK), it suggests just plain old English. And oh, it gives me the option to change my language to the real English of English (US).

Hey, I have news for you YouTube: English, English (UK), English (GB) and English (US) are all freakin' English.

Yahoo! Search Doing a SERPs Usability Survey

I was just searching with Yahoo! and I saw a survey request from "Yahoo! Surveys". It was a big purple box to the immediate right of the results list, and it was anchored to the bottom of the screen (so even if I scrolled down, it went down too). I clicked on it before I realized I should have taken a screenshot, but I did take a screenshot of the single question in the survey. The question opens in a new window:

Photobucket

Click for full size, and no, I'm not going to tell you what my answer was :p

The SEOmoz Linkscape Ghost

If you're part of the SEO industry, unless you've been livining under a rock for the past couple of days, you will know that SEOmoz launched a new tool called Linkscape, to much fanfare. First things first, congrats and kudos are due to the SEOmoz team for building such a complex beast. It's not easy at the very least on the technical level.

But there is a problem: SEOmoz has not disclosed the user agent (UA) of its crawler. Here I will talk about why this is a bad thing, and also take a stab and go out on a limb and say: there is no SEOmoz crawler, at least not in the traditional sense. For the latter, I will offer a viable technical alternative, which may or not be correct, but the fact the alternative exists gives a sensible explanation as to why SEOmoz is not offering a straight answer to the UA question.

Why Disclosing the UA is Essential

Let's not mince words: we as an SEO community like a little mud fight once in a while. We debate and discuss and yes fight. But one thing we all know how to recognize is malicious activity and differentiate it from aggressive activity.

Example: a bot scraping our content for an MFA site is a tolerated nusance. We take steps to negate the effects of scrapers but at the end of the day we don't fight them hard. On the other hand, a bot probing for security holes is treated like a witch in 1209AD.

Which is why the Linkscape's lack of disclosure hurts: We as a community work hard at identifiying bots. SEOmoz is supposed to be a good citizen of the SEO world, and yet the lack of transparency goes against the spirit and the image of SEOmoz. On the one hand we have a company with a strong community doing good deeds (SEO trademark fight anyone?) and yet it behaves in a way we expect out of the shady side of the net we deal with every day.

Not just that: the data collected from us, about us, will be used against us. It's called competitive intelligence.

And not just that: SEOmoz is using the data to make money. The free version is pathetic and the Pro version needs a monthly subscription.

To me, this kind of behavior (stealth, harmful, and to make money) puts Linkscape squarely in the naughty corner. I certainly didn't expect this out of SEOmoz. Tough luck Rand and co: you have a great brand and I for one expect better!

But I won't ask for a UA because I think there isn't one.

How To Build Linkscape

It's actually quite easy on a conceptual level. However, just like cooking, having a recipe doesn't make you a great chef - there are lots of details that SEOmoz must have tackled successfully to build Linkscape. I am not trying to belittle their achievment, and all I can show you is one recipe. This recipe is completely my guess and could very well be wrong. I have not talked to anyone at SEOmoz.

So come on Pierre, what is it? The answer is the Yahoo! Search API. It's an API giving programmers complete access over the Yahoo! index without crawling to a single page. For example, the following URL:

http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=site%3Aseomoz.org%2F&results=2

fetches the first two hits from a Yahoo! [site:seomoz.org]. Interestingly, it tells you where the cache URLs are, and they reside on Yahoo! servers (unsurprisingly). So you fetch the cache from Yahoo!, do the analysis, save what you care about (links, titles, etc), and you're done.

You'll need to kick start this somehow with a seed set of sites. DMOZ and Wikipedia are usually good sources that are freely available. Wikipedia can even be downloaded so no one needs to know. Yahoo!'s very own Delicious, Digg, reddit, etc are also good starting points because they tell you what's hot right now. The seed is basically a huge set of URLs from which you extract the domain names and do [site:domain] queries. Lather, rinse, repeat.

Notice that you won't need to crawl a single page yourself. You let Yahoo! do the work for you. Neat, no?

So What Should SEOmoz Disclose?

Above I said two potentially conflicting things: SEOmoz should disclose the Linkscape user agent and then went on to show that it doesn't need to have a user agent. So what exactly am I asking from SEOmoz?

Easy: complete disclosure. If SEOmoz is using a traditional crawler, we must have its UA and the IP addresses. It's only a matter of time for us to find them. If not, SEOmoz needs to explain clearly why not.

I Want Your Horror Stories

Ladies and gentlemen: I'm writing a post next week and I need your help. I want stories from the trenches about how developers and SEOs talk (or not...) with each other.

Comment below. If you want to remain anonymous, please let me know.

Want an example? @Harith on twitter.

Announcing Cligs: Short URLs with Analytics and SEO Friendliness

That's right folks, the short URL market is broken and I'm fixing it. The new service is called Cligs (like Clicks but with a G). It's a short URL service on steroids. The key feature is that it tracks the clicks of the short URLs.

What kind of analytics do you get? At launch right now:

  • Cligs gives you tons of traffic data and analytics about the traffic your short URLs get. This includes:
    • Number of hits
    • Referral stats
    • Mentions on twitter, blogs, and the web
    • Mentions of the destination URL on twitter, blogs, the web, and delicious
    And lots more! And if you want a more data, just let me know!
  • Cligs forwards with a 301 Permanent Redirect so your destination URL gets full SEO benefits of the link. If you are an affiliate marketer, this means you can hide your backlinks, get traffic, get statistics, and get the SEO benefits.
  • With Cligs, you can create an unlimited number of short URLs for the same destination URL. This is great because you can promote the same destination at different sites like twitter or facebook by using different cligs and watch how each source sends you traffic.

That's just the start. There are a ton of new features that are going to be added in the coming few days and weeks, including some SEO-useful analytics.

And, of course, there is a bookmarklet:

Shorten Link @ Cli.gs

So what are you waiting for? Stop using plain-vanilla short URL services and start using Cligs.

Comments and feedback most welcome.

New Stealth Crawler from Yahoo!

For the past few months, I've been tracking a crawler from Yahoo! that does not identify itself on my science blog. The bot's details are:

Requested page: /science/converting-blood-groups
  • At: 06 May 2008 10:21:05 AM GMT
  • Routed to: /index.php
  • Referred from: http://blogsci.com/science/converting-blood-groups
  • Remote: crawl1.image.srch.kr1.yahoo.com (203.212.174.181)
  • Request: HTTP/1.1 GET
  • Accepting:
    • HTTP: */*
    • Charset:
    • Enconding:
    • Languages:
  • UA:
  • Cookies:

Notice a few interesting details: No user-agent string, the fact it provides an HTTP_REFERER header that's the same page being requested, it comes from *.yahoo.com not the usual yahoo.net for Slurp, and the fact it says "image" and "srch" in the host.

The tracking is very low-level, a few hits a day with lots of one-hit-a-day visits.

What's really interesting is how laser-targeted it is: it's only requested the same two pages many times since May. The pages are the specific blog post linked to above plus the archive page that contains that post, so it's likely something about that post that's of interest to the bot. And yes, the post contains an image, and the image is the only one in the main content of the archive.

I'll dig deeper when I have a chance. Please let me know in the comments below if you're seeing something similar.

Questions Google Must Answer About Chrome

Finally I've had a chance to test Google's latest installment in the long-running series of half-baked betas, Google Chrome. Honestly, I will not rant in this post about how Google is taking over the web or whatnot, but I will ask a series of questions that I would like answered honestly and without any marketing gimmicks that are supposed to live up to Google's do-no-evil-we're-cute hype.

  • Let's start with the license. A lot of people have noticed that the Google Chrome fine-print contains some really dodgy items (examples: CNet and read write web). The offending bit is this:

    11. Content license from you

    11.1 You retain copyright and any other rights you already hold in Content which you submit, post or display on or through, the Services. By submitting, posting or displaying the content you give Google a perpetual, irrevocable, worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute any Content which you submit, post or display on or through, the Services. This license is for the sole purpose of enabling Google to display, distribute and promote the Services and may be revoked for certain Services as defined in the Additional Terms of those Services.

    11.2 You agree that this license includes a right for Google to make such Content available to other companies, organizations or individuals with whom Google has relationships for the provision of syndicated services, and to use such Content in connection with the provision of those services.

    11.3 You understand that Google, in performing the required technical steps to provide the Services to our users, may (a) transmit or distribute your Content over various public networks and in various media; and (b) make such changes to your Content as are necessary to conform and adapt that Content to the technical requirements of connecting networks, devices, services or media. You agree that this license shall permit Google to take these actions.

    11.4 You confirm and warrant to Google that you have all the rights, power and authority necessary to grant the above license.

    Ominous, no? This prompted Google's Matt Cutts to "dispel" this conspiracy theory. He went right to the heart of Googleborg and got a straight answer:

    In order to keep things simple for our users, we try to use the same set of legal terms (our Universal Terms of Service) for many of our products. Sometimes, as in the case of Google Chrome, this means that the legal terms for a specific product may include terms that don’t apply well to the use of that product. We are working quickly to remove language from Section 11 of the current Google Chrome terms of service. This change will apply retroactively to all users who have downloaded Google Chrome.

    Gee, Google, I didn't know we can pick and choose terms and conditions to apply to some products and not to others. That's just not the point of a license: either it applies as a whole or it doesn't. It says so in the terms and conditions themselves:

    20.2 The Terms constitute the whole legal agreement between you and Google and govern your use of the Services (but exclude any services that Google may provide to you under a separate written agreement) and completely replace any prior agreements between you and Google in relation to the Services.

    So please, stop playing around and provide the true license.
  • While we're on the license: Chrome is Open Source, yet the license isn't. Why not use an OSI-approved license? Yes, even for the binary. Without this, you can't claim that Chrome is really open source. Unless...

  • Suppose Chrome isn't open source. Suppose you can get the code but the binary comes (or will come) with lots of other gadgetry that Google approves of. Let's, oh, take an example of no way to block adverts. But it's open source! Well yes, it's open source if you care to download the code and know what to do with it. To the rest of us, the average Google user really, having access to the source code is irrlevant. So Google can bundle whatever it wants and no one will know the difference.

  • Speaking of licenses and adverts: what is section 17 for?

  • The interface: I don't like it personally but it's OK. It's like it doesn't want you to do anything - don't you dare find the Under The Hood options. Also, I find using the window's top frame as the tab bar to be very confusing and wrong on so many levels (the tabs are part of the window not *the* window!). The question here is: why? What logical argument places an interface element in the window border?

  • Really, when does Chrome contact Google? Matt Cutts posted about this and also Twittered to a quick privacy review that gave Chrome a thumbs up. But let's quote Matt:

    If you are typing a search or url in the address bar, Google Chrome will talk to the current search service to try to offer useful query/url suggestions.

    Search suggestions are fine - great actually - because they help me search. Querying Google (the default search service) about URLs is off limits. Let's not mince words here: some URLs *are* private. What kinds? Flickr protected albums for one. To share a private album in Flickr, it creates a unique URL that you share with your friends. No one else knows it so it's a decent enough protection for this scenario. Why does Google want to know? And does Google log it? Will the data be magically incorporated into Google Website Trends or Analytics?

So until Google figures out its license, and until Google gives us straight answers, Chrome is uninstalled from my machine.

What do you think? Comments below please.

AdSense Login System is Broken

OK, Google, I've had it with your excessive stupidities. I'm talking about the monstrosity that's your AdSense login system.

I posted about this over on Cre8 back in February. The summary is this:

  • I have an AdSense login, let's call it abc@xyz.com.

  • When I logged in back in February, it said migrate your account. So I did, but, silly me, I didn't want to change my login address so I entered abc@xyz.com as my address. It worked... until the next time I logged in.

  • Since then, so for the past 6 months, AdSense kept asking me to migrate my login, which I already did! Not only that, as the image below shows (from the Cre8 thread above), it has a message in the yellow box: Stupid AdSense login message The message says: "Are you one of the following people?" and the list below the answer is abc@xyz.com, which is the email address I just used to login.

  • The answer to this question is yes, I am abc@xyz.com, and so I follow its request to "Return to AdSense" and login using abc@xyz.com. When I do that, I enter an endless loop: I login and I get shown this error.

  • But there is - or was until earlier today - hope! See, I could skip this stupid wizard and continue to see today's crappy earnings. I've been skipping the wizard for the past 6 months. And today, they are forcing me to migrate my account.

  • And here is the kicker: I cannot tell this stupid system that I want abc@xyz.com to be my new Google Accounts login for AdSense. Why should I have to create another account? Listen, Google, if you want to inflate your accounts count, I'd be very happy to create a few more for you. I even won't tell your stockholders if you don't.

  • So I thought I'd ask for help. The link in the top right ends up here and I click through to Contacting AdSense, then to Do you have a customer support telephone number? which says "no" (surprised?) and suggests that I check out the AdSense Help Forum which 404s. Tada.

So let me say this as politely as possible: I find it disgusting that a company that creates products with such a crappy user experience is considered a leader in our industry.

The Ultimate jQuery Development Guide

This has got to be the best jQuery development guide I've seen. It's one of those pages you bookmark or add to your scrapbook for those late night hacking sessions when things go wrong.

Opt Out of Behavioral Ad Targeting by Google/Doubleclick and Yahoo!

Oh yes, finally a way to tell the algo-borgs at Google/Doubleclick and Yahoo! that they should not track your behavior to deliver "more relevant" ads. You do that by visiting a page on each of their websites and click a button which sets a cookie that tells the system to not track your behavior.

Google also links to another page from the Network Advertising Initiative which lists quite a few ad systems you can opt out of.

The pages are:

While I'm at it, does anyone else find Yahoo!'s page to be much better than Google's? Think about the usability: it tells you if you've opted in or out and explains that it's per computer rather than per user (very important!!!). I'm just saying that as a landing page supposedly to help consumers, Google's is a mess compared to Yahoo!'s clean and to the point page. The NAI's is very good too.

Chatting with a Google Street View Driver

Note: some details in this post have been skipped or generalized to be a bit vague to protect the identity of the Google Streeview driver.

Google Street View Car

Sometime in the past few weeks, I was walking with a friend when we spotted a very funny looking car. We both immediately knew what it was and as the car drove closer by, our suspicions were confirmed: it was a Google Streetview car outside London. Feeling naughty, I shouted at the car as it drove by something along the lines of "there are privacy laws" and to my surprise an old man across the streed did the same! It was very funny how both of us knew what a Streetview car looked like!

Then it hit me: the road we were on that the car was driving into was a dead end road. Picture time! So I dropped my stuff and asked my friend to watch them while I set up my phone and found a good spot to take some photos as the car drove back out again. So I watched as the car reached the end, did a U-turn and drove back out again. However, as it got close to me, the car pulled up into an empty parking spot and the driver came out. He shouted at me saying "I know you want to take pictures but I don't want to be in them." I obliged.

While taking the photos, I talked to the driver a little bit. Here are some details from the notes I scribbled afterwards:

  • Google has a centre in Milton Keynes where this operation was based in. The drivers just showed up for "a driving job" (his words) and didn't know it was for Google until the arrived to pick up the cars.
  • The drivers were given training to use the computers inside the car. It's not hard: it's a large-ish touch screen (I guessed about 17in or maybe a 19in when I saw it) with a record and a pause button.
  • The screen is to the left of the driver in the passenger seat with a large server at the back in the trunk. The back seats of the car were removed - it was just a big space. The connections into the server were just power and ethernet. The ethernet seemed to be going up to the camera but I'm not sure if it ran to something else.
  • The camera is rain sensitive. It collapses in a very funky way and has to be covered. The drivers are under strict instructions to do so.
  • This particular driver was very sensitive to the privacy issues. He was having a personal conflict about the whole thing and was stopped by (his words) "10 people" that very day. Why? Because only recently had the BBC published an article about Google Streetview starting with Google's plans to launch a mapping tool in the UK could be referred to the Information Commissioner". No wonder the driver didn't want to be in the photo!

Now some photos of the car with notes:

Google StreetView car, front view

The car from the front.

Google StreetView camera

The car's camera. The hexagon Octagon at the top is I think is the camera set itself (so 6 8 cameras in total). The yellow box seems to be the communication/processing circuitry; the yellow box is on the back side of the car and so the white box thing at the right hand side of the image points towards the right of the car. This white box thing seems to swivel up and down but this is just a wild guess.

Google StreetView car, back view

The car's camera kit as seen from the rear of the car. Just guessing what each bit is: Yello box at the top, as above. White boxes to the left and right are the (potentially) swiveling bits - could they be cameras? The yellow disk at the bottom: a wireless communications dish? It could be a GPS receiver.

Update: Looking through some of the other images I had after someone dropped a hint on GTalk to me, the white boxes under the hexagon of cameras are laser range finders. Sure enough, I have a photo that has a warning that it's a "Class 1 Laser".

Update 2: Thanks for all the comments. Yes I couldn't count: there are 8 cameras not 6; that's fixed now. Also, a lot of people wrote about the type of laser range finder and why you'd need it - see the comments below. Finally, lots of people noted a certain irony in the driver not wanting to be photographed. Point taken, but the guy was very conflicted about it. The BBC article was still in memory and clearly some people like me caused his some fuss on that day. He was talking a lot about wanting to quit this job. Deep down I think he did but of course I cannot know.

Update 3:Yes some rain droplets is visible in a photo. It wasn't raining while we were talking but it had rained earlier that day. When the driver parked, the camera hit some trees (you can see that in the photos) and the droplets are from the tree. It's hard rain that gets the equipment as I understand it, and that's when the drivers are supposed to cover up.

Twitter Bug: View Friend-Only Private Updates

On twitter, I'm following someone who I cannot un-follow due to a bug in Twitter. Why? Because said person changed their settings I'm only giving updates to friends - I see the message "I'm only giving updates to friends.". Visiting the person's home page, I cannot see the Follow/Unfollow button because the interface only lets me ask the person to allow me to see his updates.

But I can easily see his updates.

Here is how: browse twitter using a mobile phone. Yes the mobile interface shows you these "private" updates but the web interface shows me the message "I'm only giving updates to friends.". I discovered this bug by accident while browsing using my mobile phone, but using a couple of extensions, you can pull off this trick in Firefox.

The screenshot below illustrates the bug. It's basically the mobile version and the full normal version of twitter side by side. The lines map corresponding updates, with the yellow/orange one highlighting the bug.

Twitter bug showing private updates

Download full sizes of the screenshots used to make the image above:

I've filed a bug report with twitter.

What do you do with Unauthenticated Search Engine Bots?

Over at Search Engine Journal, Ann Smarty explains how to switch your UA to Googlebot and browse the web. The technique uses a Firefox extension to change the user agent string to that of Googlebot. Simple and works a treat. Except for...

The problem here is that it is very easy to authenticate Googlebot, Slurp, or MSNBot. The three major search engines give us a double-DNS trip to check whether a request pretending to be one of their crawlers is genuine or not. The authentication helps us webmasters fight against crawlers (not to mention other things ;) ). So the SEJ article is useful but it's not 100% foolproof and people pretending to be GBot/Slurp/MSNBot will probably get trapped with snares laid by clever webmasters.

This raises an interesting question: If you do authenticate SE bot requests, what do you do with unauthenticated ones?

Personally, I just block all unauthenticated bots. The request is served with a blank page without any content. I've found that this helped stop *all* (yes all) unauthenticated bots but with proportional rise in more sleuthing bots (i.e. scrapers pretending to be a browser). No matter, this is an arms race and I'm in it for the long-run.

Other people suggest you should feed unauthenticated requests with content that AdSense frowns upon like guns or porn. The idea is that these crawlers are out to get your content for MFA sites and so it's best to get them banned the quick and dirty way.

Others suggest just ignoring them; after all, they'll come back with a different UA anyway, so what's the point? This attitude bothers me because it just means giving up and letting your content get scraped far and wide without any control.

So what do you do with unauthenticated bots and more generally, what do you do with bots?

Stop Competitors from Stalking Your Website Using AdWords

Regular readers will know that I like to gaze at my log files in search of life-changing inspirational moments. Well I have another such gem of an inspiration for you: figuring out if someone is stalking your website using the Google AdWords keyword tool and how to stop them.

When someone goes to the AdWords keyword tool and asks for keywords based on the contents of a web page (the "Website content" option), Google actually requests the page live. This request shows up in the logs and can of course be blocked. The details are:

Referred from: (No referer.)
Remote: 74.125.16.37
Request: HTTP/1.1 GET
UA: Mozilla/5.0 (compatible; Google Keyword Tool; +https://adwords.google.com/select/KeywordToolExternal)

So what to do? Be careful blocking the IP addresses as a general precaution against stopping legitimate requests from Google IP addresses (Googlebot, Google's Feedfetcher, etc). However, the user agent is a good tell-tale sign and is ripe for blocking.

So: aim... fire!

Fire what though? A simple block? Nah, not much fun that. Knowing full well that only competitors will use that service to check out which keywords your pages might rank for, I would feed the requests dud content. Lorem ipsum anyone? How about random content about keyword theft? Here is an SEO exercise for you: which keywords can you get the Adwords keyword tool to show about your pages? To rephrase: what keywords can you "rank" for in the tool?

And don't forget to go back into your logs and see how many times people have stalked you.

Google Down in the UK

Various reports of Google being down (including for me) here in the UK. It seems to be a few datacenters so it works some times but mostly not. Reports also talk about G Docs, Mail, and YouTube. Sometimes a redirect to google.co.uk works, but mostly even that fails. Some people can get to google.co.uk if they browse to that directly. I've been logged into GMail since last night and it works OK.

Seems to me that there is a DNS issue at play here in that if your browser requests a fresh IP resolution, it works, but the IP addresses fail. If your browser has an IP address cached it seems you're fine.

Reports:

Anyone else seeing this?

Broken GMail Login

This is becoming more frequent so I thought I'd mention it: Once in a while, I can't log into GMail. It started on Gecko-based browsers on OSX; switching to Safari invariably worked. Then it started happening with Safari, and now it's broken on my Ubuntu machine using Seamonkey and Opera.

There are two ways it breaks:

  • Eternally looping on the loading progress bar. What happens is that it keeps refreshing the loading page and never making it to the email list.
  • Mostly on Opera, it just returns to the login page although the username and password are correct; it doesn't show any error messages.

I've learned to be very quick at clicking the the simple HTML view link, but even with all the practice sometimes even that doesn't work.

Anyone else seeing this?

« Previous Entries  

Site Navigation

Blog Categories

Popular Pages

The most popular pages on eKstreme.com.

Search

Subscribe

Subscribe to RSS 2.0 feed