I've had it. The live.com spambot, aka msnbot, is officially not welcome either here or at Social Alerter. Why? The bot is still referral spamming. How much? 100% of my live.com referrals at Social Alerter are actually the bot's spam. Granted the absolute number of hits is only in the low tens, but it is not right and such behavior is no longer welcome. And no, the constant lies that this behavior has stopped do not help.

For a background on this, start here, then read this post, and close off with the follow up.

Bye, bye. I hope to see you never.

This kind of post is something a few bloggers do. I enjoy reading them, so I thought I'd try my hand. It's a bit of the final score-card for the year and hopefully inspiration to do better (whatever that actually means) for next year. So what happened with me in 2007?

January

January is probably the best month of 2007. It kicked off the year with my first ever digg home pager, me doing a live podcast/talkshow, and the first rant of the year which set the pace for the months to come

It wasn't all fun and joy though: in January, eKstreme.com suffered a DoS attack.

February

February brought lots of developments: I started moderating at Cre8 a Site Forums, easily the friendliest place on the net. The second Digg home pager arrived too, and a major statistical analysis of the Socializer data got a lot of people interested.

March-July

Very quiet period. In March, I was busy thinking about my online strategy about eKstreme.com, blogSci.com, and the other major property I owned back then, fontfox.com. The outcome of that is a major change (for the better!) monitization effort of eKstreme.com, a decision to keep blogSci.com ad-free, and realizing that I wasn't doing much with fontfox. In the end, fonfox got sold in May.

In July, this blog got its first ever guest post. It was a great piece. However, this effort to bring fresh blood into this site was a dud: a lot of other people agreed to blog post but none actually sent me stuff Waaah.

Of course, lots of ranty anti-Google posts were written in this period. Back then, Google thought it was OK to abuse user data in many ways. To this day I still think they are abusing our data and it will probably get worse in 2008.

June-August

While the blogging was quiet, a lot was happening in the background. The CMS of eKstreme.com has been showing its age and slowing things down. The strategic review in February concluded that this has to be fixed. So the whole site was moved to use Wordpress as the CMS, which involved a lot of hacking to get WP to like my SEO tools and not break them. I also moved hosts.

July onwards

I started taking a very close look at the bots/crawlers hitting eKstreme.com and blogSci.com. This research resulted in a lot of bot-related posts and insights. I'm still collecting data to learn more about how bots look like. By bots, I mean the more malicious scraper spammy types, not the nice ones like Googlebot and Slurp!.

Out of this also came the realization that msnbot was misbehaving. First, the authentication was broken, second, it was not obeying the robots.txt file, and thirdly, a very strange pattern of bot activity from live.com was detected. This resulted in third Digg home pager. A few weeks later, MS backtracked. I don't know if it had anything to do with my post or not - I doubt it.

All in all, a great year. Stay tuned for 2008 because there is a lot of great stuff coming. They'll be announced here as always.

Predictions for 2008

Now the really fun part What will happen in 2008? Here are some of my predictions:

• Online office: Microsoft will release Silverlight 2.0 in early '08 (we already know that). Shortly afterwards, they'll release an online version of Office based on that. This will disrupt the market, making Google's Apps look like toys and Zoho very very vulnerable. Zoho will get acquired.

• At least one major privacy scare on the web. Top contenders are Google and Facebook, but Microsoft cannot be discounted. My prediction is that it will be related to user profiling for ad-targeting purposes.
• Rich Internet Applications (RIA) will arrive in full force. Everyone will look at each other and go 'eh' until a killer app is released. That app will probably be the online MS Office. Top contenders are Silverlight and Flex from Adobe. Flex has no chance against Silverlight because Adobe doesn't know how to write web-friendly software (like Acrobat Reader plugin for browsers, which sucks) and certainly is no match for the developer-friendly MS. Flex will live through 2008 though because end-consumers will think it's the Flash player.
• In Search: Google will continue to dominate, but slow down its growth. Semantic search engines like Powerset (which I'm a member of the public beta testers) will rock. Hakia will figure out that its biggest obstacle to world domination is its index: full of spam and very stale. Their technology is great though.
• Yahoo will chug along. A few gem products will come out of their R&D efforts along with the continuous stream of half-baked ideas. The new delicious service, which finally loses the an.nno.ying dots from its name will be a great hit.
• Generally: more memes and more bloggers working in synchrony for a common cause.

So... will I eat my words in December 2008? Stick around and you'll find out

Yesterday, I posted my 2000th post on Cre8asite Forums where I moderate. The post was about writing tools (again!) but this time with a twist: the tool is released under a BSD license, meaning it can be downloaded and used as you see fit. The project, called Open Keyword, is a keyword generation tool for SEO purposes.

Open Keyword has two components:

• A keyword searching tool that gets related keywords from Google Suggest, Yahoo! Live Search, and Yahoo Related Search.
• A keyword scraping tool that gets the list of keywords given by Google Trends Hot List that's updated hourly. These keywords are stored into a database and retrieved by the search script.

There are two key things about Open Keyword that make it unique:

• You run it yourself, meaning you can customize the code, for example to alert you when a keyword has become popular. Also, no one will be able to log your keyword research activity (as would happen if you use tools on other sites) and because you have the data, you can do funky custom analysis yourself.
• Because of the data sources, you know the keywords are the most popular according to the search engines. As such, they can be used as seeds for further keyword research to build up the keyword list.

And since Open Keyword is open source, anyone can write improvements. If you do, please share them with me and I'll incorporate them so that everyone benefits.

So all you have to do now is go to the Open Keyword home page and download it. Full instructions on setting it up are also on the page. And if you get stuck gimme a shout. Comments and thoughts below please

A quick update to the Socializer:

• Mister Wong has been added to the services, in the Top Services section. Why? They're very large in Europe with multi-language support and a dedicated user base.
• Netvouz was moved into the Top Services section. I did a quick check of the rankings (like this) and it certainly deserves it. Heck, it out-ranked Slashdot! I've kept Slashdot in the Top Services section because of their new Firehose service. I'm going to be keeping an eye on it.
• The web-marketer's Digg, Sphinn, has been added to the list. It certainly a niche service, so it's not grouped as a Top Service. However, I'm going to be keeping a very strong eye on it and PlugIM. Depending on the rankings, a switch might be in order. We'll see.

We now return to our regularly scheduled silence Yes, I'm very sorry for the lack of posts but I'm working on a new service that will launch shortly. It's been taking up quite a bit of time. More soon - like this weekend soon!

As you may have noticed, eKstreme.com now sports a brand new design. Not only that, it's also been ported to a new backend, namely WordPress. On top of that, it's on a new host now that promises to be be more reliable and a whole lot faster.

Yep, a hat-trick of news that's been 3 months in the making. If you ever wondered why I haven't been replying to emails much, or why I haven't been blogging much lately, this is why. A lot of effort went into this, and I'm sooooo relieved it's over... almost.

Things will be broken! I know that. Firstly, the tools are complex beasts to run completely in WP. Yet they do, for the most part. Two tools are broken at the moment: the spell checker and the Backlink Social Celebrity tool. The first one is waiting on some software to be installed on the server, so it might take a few days. The latter is just sub-par coding on my part and I decided that instead of trying to fix it, I'd re-write the thing. Rest assured I'm on the case and I'll have them up ASAP.

Having said this, please file bug reports and feedback using the contact form.

Speaking of which, there are two people I'd like to thank wholeheartedly:

• Joe Dolson who patiently gave constant feedback on the design as it was evolving (slowly) and helped troubleshoot some DNS issues.
• Mike Cherim because eKstreme.com now uses the WP port of his contact form. It was very easy to add it in and it works a treat.

So, thanks to all the readers and users of eKstreme.com. This redesign was done with you in mind. As always, feedback welcome, and there is a LOT more to come now!

In the past few weeks I saw the same question asked twice: how do I choose which social bookmarking sites to use on my blog or website? Here is my explanation of how different sites do it and my suggestion of 'best practice'.

There are three approaches I've seen on the web:

• The 'Link Series' approach: a series of links, sometimes with images, dumped on the page. Honestly, I think this is a waste of space as the CTR of these links is minimal. This is what I had on eKstreme.com and what led me to create the Socializer in the first place. If you go down this route, keep an eye on what's going. I suggest you hack the AJAX link tracker.
• The 'Chosen Few' approach which links to only a select few of the myriad of social sites out there. If you're a shopping site, you can submit to social shopping sites. Science-type sites can use Conntea, which is a delicious-type service for academics. Techie sites go for digg and slashdot. You get the idea.
• The Socializer approach, which you know well. Some people use it on its own, and that could be the wrong way to do it. There are two really good ways to use the Socializer:
  • If you use just the Socializer, user a very descriptive anchor text. For example, on my Science Blog, I use the anchor text "Social bookmark post (digg, delicious, reddit, etc)". I also use the Socializer logo as I've noticed having the image increases CTR.
  • Another excellent variation I've seen is that you link to very few sites (say, delicious and reddit) and then use the Socializer as "the rest" link. I like this approach, as it gives easy access to the social sites your visitors use most often, but covers all bases.

For a website that doesn't have an unusual layout (like a blog), I recommend the last variation. However, test, test, and then test some more!

The real question is how do you choose which services to list? For each site, test the various services. However, perhaps the strongest hint is your log files: which services refer hits to you the most? These are these are the services your visitors are already using. This data is even more interesting if your site does not already have social bookmarking links; in this case, the log files tell you the most popular services your visitors use without any bias from your help.

Finally, a comment about location. I think, although I haven't tested this extensively, is that you should place your socializing links in a consistent manner throughout the site at the bottom of each page or blog post. I've also seen some very successful links placed at the top of the pages. Also, my experience is that using the services' logos helps the CTR a lot, in addition to descriptive text if you can squeeze some in.

And there you have it. Questions or comments below

Technorati Tags: socializer, social, bookmarking

Tomorrow, the 22nd of February, marks the first birthday of me officially announcing the Socializer for public use. For a few months before that, the Socializer was in use (and thus testing) on eKstreme.com.

To mark the occasion, the Socializer now has new design to bring it more in line with the Web 2.0 look. Please tell me what you think of the new layout either in the comments below or over at the site review thread at Cre8.

So what does the future hold? There are many, and I mean many, updates on the way. I don't want to spoil the suprise, so I'll keep mum for now, but webmasters will be very interested in what's coming. I can't promise a time line as I'm so busy, but hopefully within a month things will be launched into public testing.

This past year has been amazing for me. eKstreme.com's traffic went up 10x. I met some great and wonderful people. Even a little community of sorts coalesced. All this was mostly due to the attention the Socializer garnered. Never did I think that it will become as popular as it did, and I want to thank everyone who blogged about, who suggested to friends, and to its users for making it what it is.

So thanks again to everyone, and if you have any feedback, recommendations, suggestions, anything, please just give me a shout.

Technorati Tags: Socializer

A thread at Cre8 asked a simple question: how do you monitor your brand on the web? To rephrase, how do you keep track of what is being said about a certain subject?

As you may know, eKstreme.com is home to a tool to do just that, called What the Buzz?. You enter a search term (your brand, your favorite topic, anything) and you'll then get a large dataset: blog popularity chart, Google Trends, tagged blog posts, Topix.net search, blog posts containing your search (via Technorati and Google blog search), and social bookmarks tagged with your keyword. The Google blog search is a new feature uploaded just now.

So go really check your keyword popularity. Note that the URLs are bookmarkable so you can always come back to get the latest very quickly!

Technorati Tags: keyword popularity, brand monitoring, ekstreme.com

What is the most popular social bookmarking site? There are many metrics to measure this, and one of them is submissions of links. This website is the home of the Socializer, a multi-social bookmarking service. Using data collected from visitor clicks using the Socializer, I ranked the popularity of the services it lists.

Methodology

Hundreds of sites use the Socializer for their social bookmarking needs. For each page, they create a submission link to the Socializer. When visitors arrive on the Socializer submission page, they are presented with 47 links, each being a specially formatted URL for submitting a link to one social bookmarking service. The 47 links include the most popular services like digg, del.icio.us, and reddit, and many niche and less popular services. The services are listed alphabetically in a rectangular grid, starting with Backflip on the top left corner. Here is an example.

When a visitor clicks on a submission link, the click is logged using the AJAX Link Tracker with the help of a server-side script. Because it is using AJAX, the visitor's submission is not interrupted; i.e., the data collection does not interfere with the actual bookmarking activity. Since the Socializer page contains other links than just the submission links, clicks on those are also recorded. Example of non-submission links are navigation and links to external sites that host plugins for getting the Socializer to work with Wordpress, MovableType, and others.

A few weeks ago, I reset the click logs and started counting from scratch. As of 11PM last night, I had collected 21835 clicks. Overall, these clicks represented 70% of all clicks on the Socializer's page.

Results

Digg was the service most submitted to. Digg collected 1220 clicks (5.6%) during this time. This represents a very wide lead over Netscape, the second most popular service, which collected 1009 clicks. In third and fourth positions are Yahoo!'s two properties, MyWeb 2.0 with 981 clicks and del.icio.us with 942 clicks.

The rest of the data is presented in the table below and graphically.

(Click for full size version.)

Analysis

There are many points to make here:

• Digg is very popular indeed. It has a full 21% gap over the second place service, Netscape
• Speaking of Netscape, everyone yawned when they moved to their new Digg-like service. They seem to be doing quite well to me!
• The data agree with a recent post by Rand Fishkin. He talked about links gained from social sites, and Netscape is up there with Digg, del.icio.us, and reddit. That's another metric of popularity of social sites, and it's good to see some agreement.
• There is a very tight pack trailing Digg composed of Netscape and the two Yahoo! sites del.icio.us and MyWeb 2.0. The difference is minimal, but real as far as I can tell: For the past five days, I've been checking the data, and the rankings didn't change, but they all gained clicks (as expected). It will be interesting to watch the top four in the future.
• But, but, but: although del.icio.us and MyWeb 2.0 are distinct services, together they tell of a very powerful Yahoo! in the social bookmarking market: Combined, they garnered 1923 clicks, which is 8.8% of the collected clicks. Yahoo! is the true leader here.
• Notice the green trend line I've plotted on the graph. It's a best fit logarithmic trend, and the fit is really good. This harkens to the idea of a 'long tail' where we observe very few categories (in this case social bookmarking services) being the most popular followed by a long tail of categories of ever decreasing popularity. This is interesting and warrants further study.
• We have to stop and think about toolbars. The key service here is StumbleUpon which shows a very paltry performance. However, most SU submissions are probably done using their toolbar that users install, and so we expect to see very few people using their submission URLs.
• One service I'm surprised to see perform so well is PlugIM. I've mentioned it several times in previous blog posts as it's relevant to internet marketing. I asked the creator of PlugIM to give me exact details, and he said that PlugIM officially launched September 1st of 2006, a mere five months ago. In this light, they're doing really well!
• The performance of Backflip amazes me again. Last time I did this league table, Backflip also performed very well, but frankly, I don't hear much about it on the web! One suggestion is that since it is the top left link, it is the first link visitors see and click it anyway. This is a bit like putting ads in the top left triangle area of a web page. I accept that this is a real possibility, but I'll study it more before making judgement.
• We have to note again how the data was collected, namely, using Javascript. Although not all people have Javascript enabled, there is no reason to believe that those that do and those that don't behave differently from each other. I'm very happy to accept that users browsing with Javascript enabled are a representative sample of the Socializer's users in general.
• Finally, we have to look at the demographics of the Socializer's users. The sites that use the Socializer are diverse and target very different markets. The markets include technical sites, adult sites, commercial sites, and charities. I have no reason to believe that the sites using the Socializer are not representative of websites in general. The largest category of sites is blogs, though, but I don't know the exact percentage.

Future Work

Well, this is the second time I do this ranking, and the results are very interesting. In a few months time, I'll quietly reset the counters again and do the experiment again.

Also, there is a new version of the Socializer in development. It will have many new features, some aimed at gaining better understanding of the how users use social bookmarking services. The data will be published like this routinely.

In the meantime, I've partnered with SEO Refugee Forums using a customized Socializer, and I'm looking for other partners too. No data has been collected, but it will be interesting to see if the global usage pattern holds for individual sites or not. Should be fun

Any questions or comments, please comment below or email me.

Yesterday was crazy. At one point, eKstreme.com went down but my other sites on the same host remained functional. Eh? After investigating with support, turns out there was a SYN flood attack being carried out against the IP address of eKstreme.com. The IP address was dedicated to this site only.

While we were figuring out what's going no, life threw us a curve ball: The datacenter hosting the site went down. Although it's unclear what exactly went wrong, it seems to be that the power unit of the core routers failed and took down the routers with it. So now we had two problems to tease out. Sweet.

To fix the situation, support used the firewall to block all incoming traffic to the IP address. Simultaneously, I changed the IP address of eKstreme.com. This means that now we have to wait for all DNS servers around the web to update, and so the site will appear down for people until the DNS propagation reaches them. To speed things along, my host issued a DNS flush. The power problem was fixed within an hour.

In short: apologies for the downtime!

The other thing I want to mention here: I want to thank the support team at my host, SLHost. Their dedication and quick, knowledgeable responses ensured that any damage was minimal. Thank you! Although the site went offline (and in a twisted way, the DoS attack succeeded!), it could have been much worse.

So what's a SYN flood anyway? It's actually a very simple kind of denial of service attack. To understand it, we need to delve into how connections are made between computers. The process starts off with a computer making contact with the server. This initial connection is called the SYN packet, somewhat akin to you starting a conversation by saying "hello". The server responds then says 'ooh, this computer is trying to contact me, so I'll add that to my list of computers trying to reach me'. This list is a queue of sorts, and once a connection request joins the queue, the server sends out a reply packet, called the SYN-ACK packet; ACK means acknowledge. To continue our analogy, this is akin to someone replying to your hello with "Hi, I'm Pierre". When the connecting computer receives the SYN-ACK packet, it sends back one last packet, called the ACK packet. When the server receives an ACK packet, it looks in its queue, finds the details and the connection is established.

The SYN flood is basically interfering with this 'three-way' handshake. The attack is essentially a flood of SYN packets with spoofed IP addresses. When the server sends the SYN-ACK packet, it waits a bit for the ACK packet to come back. After a while, if no reply was received, a timeout happens and the request is removed from the queue. This wait is the weakness: the flood of SYN packets quickly fills up the queue so that no new connections get accepted. And because the IP addresses are spoofed, the server will be sending SYN-ACK to IP addresses that will never respond with ACK (because they didn't try to connect, so they ignore the ACK packet). So because the queue is full and the server is just waiting ignoring new connection attempts, the server appears offline.

So, apologies again for the unexpected downtime. The server itself is fine now. If you spot any problems, please contact me. Thanks!

Great two additions to the Socializer: PlugIM and StumbleUpon are now part of the Socializer. The grand total now is 47 bookmarking services, all available with one click!

PlugIM, in particular, is a particularly cool and important addition. From their website, PlugIM is described as a "user driven internet marketing community". Essentially, it's the Digg of online marketers, so better keep an eye on it! My username is pierrefar, so you can add me as a friend

Also, in conjunction to adding PlugIM to the Socializer, they have a series of social bookmakring links, and now the Socializer is one of them. It's a great collaboration, and I'm happy to see them already reporting new sign-ups coming from the Socializer!

Technorati Tags: socializer, plugim, stumbleupon

This is a great idea that Eric Ward suggested last week: wouldn't it be great if the Socializer URls were short so that you embed them in press releases? Well, yes, of course. He's absolutely right, and I'm happy to announce the launch of the Short Socializer service.

There are two simple ways to use the Short Socializer feature:

• Go to the Short Socializer home page, type in a URL and a title and a short URL corresponding to the long Socializer bookmarking URL is created for you.
• Visit the full Socializer bookmarking page. Below the bookmarking links is a link to the Short Socializer service to create a short URL for you.

As with the Socializer, the Short Socializer is 100% free!

Please test it, use it, and tell me what you think! Thanks!

So what happens if you drop off the face of the Internet for 2 months? For me, I came back to find over 4600 blog posts to read and actually had a guy verbally abuse me because I haven't responded to his email (apologies to him and everyone!).

So here is a quick summary of what I've read/done so far catching up:

• Netscape.com has been added to the Socializer and to the Social Bookmarking Code page. Thanks to the nice guys over at Netscape for contacting me and provding the code - I didn't know they had auto-submission URLs. So go there and have a play with this Digg-like site. Again: please send me the code for the social bookmarking sites you use!
• On one of my other sites, late in August I announced the launch of the Daily Mental Calisthenics. It's a cool PHP script I wrote, but the real news is the response from the StumbleUpon community. Traffic-wise, you really can get swamped if you have a page that appeals to Stumblers. I'll publish a case study about this when I get a chance.
• Over at SEO Refugee, they have a very cool piece called Poker Lessons on Affiliate Marketing. If you don't get what it takes to be an affiliate, this should help.
• Want to be a better writer? If you're a blogger or an SEO copywriter, you better want to be! Escape from Cubicle Nation has a great post about resources about writing. Read them for better writing.
• I just learned that Yahoo! has a browser-based authentication system so that you can have people log into your site (and SEO tools... ahem) using their Yahoo! IDs. How did I learn this? Why from Matt of course!
• MANY link-building tutorials, nicely formatted into step-by-step howtos and commandments (eh?). Anyway, in no particular order:
  • Four Trusted Links You Can Build Today
  • Secrets to Beating the Sandbox 2.0 REVEALED: The Ultimate Guide. This one is a keeper.
  • Link Building Cycle
  • Jim Boykin’s Favorite SEO Writings
  • The Google Gods Speak to Us about Link Building
  • The 10 Commandments of Link Building
  • Linking Commandments - Part I

Plus: I got an idea for a very cool SEO tool. This one will be a gem to add to the collection.

So why the two month drop? Simple: I started work in the real world, and where I was staying, we didn't have net access in the evening. Surprisingly, I enjoyed the peace and quiet

One of the lesser known gems (and my favorite tool!) on eKstreme.com is What the Buzz?. This tool helps anyone keep their finger on the pulse of any topic. Simply feed it a keyword, and it scours Technorati, Google, and del.icio.us for everything related to that keyword. In short, it gives you a comprehensive and deep look into the buzz surrounding a topic.

Today's update brings two sorely needed features:

• What the Buzz? now searches Topix.net for more info about your keyword.
• The results are now formatted in a cleaner fashion. The old way was just horrible and dirty compared to what it is now. Granted it is still a huge list dump, but at least it's easier on the eyes!

So go use it, break it, suggest features, and best of all: bookmark it. Conveniently, What the Buzz? searches can be bookmarked.

Technorati Tags: buzz, eKstreme.com

Today, Yahoo! annouced their latest tool, the Yahoo! Search Builder. It's quite a funky tool that sort of steps over the rollyo turf. It allows you to build customized searches you can use on your websites.

This caught my interest, so I added to eKstreme.com, to show up on every page (apart from the home page). The addition also redesigned the Socializer link to make it tidier and to implement the link in the best converting manner I've seen. It looks something like this now:

Any effect on AdSense conversion will be monitored very closely, especially since this is the spot that AdSense has been on for the past few months. We shall see.

One cool feature of the Yahoo! search is that it tracks traffic and displays the most popular searches. The traffic feeds my data-junkie mode well, and should quickly tell me if it's worth keeping it or not. If it does get used a lot, maybe I'll replace it with AdSense for Search

I'll post at a later date with some feedback...

Technorati Tags: Yahoo, Google, Search

The Socializer has been updated to link to more social bookmarking systems. Total count now stands at 45 services!

Welcome to Things of Sorts, the brand new, super green (?), fun (???), blog of eKstreme.com.

What will I talk about here? Pretty much anything web-related goes here. Everything from PHP, HTML, AJAX, MySQL, etc, all the way to web design, usability, SEO/SEM, and everything else. Of course, if I find something not-so-related and oh-so-interesting, it'l be blogged about.

You can contact me at any time. Nice having you around

Pierre