Google gives a sneak peak of its updated search engine infrastructure

On August 10th Google revealed a “secret project” it has been working on – Google Caffeine – and people can even take it for a test drive before it’s live.  Slated as a new and improved infrastructure, Caffeine promises to improve the “size, indexing speed, accuracy, comprehensiveness and other dimensions” of Google’s index.  These improvements are said to be “under the hood,” meaning that “users won’t notice a difference in search results.“  Matt Cutts of Google elaborated, saying, “most of the changes are in things like our core indexing, so there’s less changes for things like rankings.  Lots of users won’t notice a big difference.”

From reading those excerpts and looking at the name “Caffeine,” it’s clear the brunt of the improvements center around the speed and comprehensiveness of the Google index, not the ranking algorithm.  However, Google did mention rank and said power users might notice some differences, so there is still reason to be concerned about rankings.

The effects on Avvo

To see the effect of Caffeine on Avvo, we ran hundreds of queries and compared the results to the new and old Google.  The results were that the rankings were only slightly different on average, with certain types of searches affected more than others.  However, for a small number of keywords, there were some very large differences.  Regarding indexing, Caffeine seems to have indexed about 6% more of Avvo’s pages than the old Google.

But why is Google focusing on its index under the hood instead of on the front end?

While it’s interesting to look at ranking changes in the “new” Google, there are some bigger picture issues worth thinking about – like why is Google so interested in indexing speed? What is Google ultimately trying to do? And how does this all affect your marketing strategy?

Real-time search has been Google’s Achilles heel

One of the most obvious reasons that Google is concentrating on speed is because real-time search engines like Twitter and Facebook have begun fulfilling a need that Google has not.  Namely, while Google is the king of general purpose search, one weakness it has is that it can take days or weeks to crawl new web content and update its index.  Granted, Google is much faster than it used to be, compared to its monthly Google Dances in the past, but it’s still not real time.  And for users who want to hop online to see what’s happening now, they’re increasingly going to Twitter because Google is too slow.

In fact, Google’s lack of real-time search is one of the main reasons for speculation that Google (or other search engines) should acquire Twitter.  This is also probably the main reason why Facebook is getting into real-time search.  However, if Google can successfully make its crawling and indexing so fast that its core search becomes real-time, then Google largely cuts off its competitors at the knees.  Caffeine does not purport to make Google a real-time search engine, but it’s a step in that direction.

So how does Google’s move toward real-time search affect your marketing strategy?

Online marketing, particularly SEO, used to be relatively simple.  Google results didn’t change often, so success meant simply getting your website to rank for a small group of high value keywords, and once you achieved your rankings, they remained fairly stable.

Nowadays, it’s much more complicated, and victory is fleeting:  Rankings change on a daily or hourly basis.  Results can be automatically personalized.  News, pictures, videos, blogs and forums can bump off your website.  People can filter results to only see forums, reviews or new items.  Keyword queries are even becoming longer and more diverse.

This latest push toward real-time search is just one of a long list of changes that are forcing website owners to think beyond simplistic keyword rankings for static websites.

All the more reason to evolve

For those looking to market themselves online, it’s important to see the forest for the trees — either start expanding your web presence or watch yourself become increasingly less visible.  Caffeine is another piece of writing on the wall.

Antiquated web strategy that centers on old-school SEO, where the object is to have your one, static website appear highly in Google for a handful of high-value key phrases, is going extinct slowly but surely.

Instead, having a dynamic presence across the web using a variety of channels is becoming what search engines are favoring, and so should you.  In fact, perhaps the largest benefit of diversifying your online presence is not that it will please search engines, but make you less dependent on them.

This entry was posted on Thursday, August 13th, 2009 at 4:42 pm by Nick and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.