Hansel and Google

November 30th, 2009

I want to draw your attention to a very subtle-change in Google SERPs: breadcrumbs.

Google has been indirectly advocating their use for quite a while now because they provide a better user experience, and an easy navigational path for their bots.

I have been urging clients to use them for several years because they provide an easy way to add keyword-rich backlinks to every page on your site, no matter how deep a given page is within a site. They also help to establish a hierarchy of content, which really helps to get search visitors to the right page from the right search.

SERPs breadcrumbs for search of 'peppermint stick old fashioned recipe'

As you can see in the screenshot above, Google has decided that breadcrumbs are so important, that they may begin using them in their regular Search Results. Certainly, this is a more human-friendly method of communicating hierarchy than filepaths.

But what effect does it have on Click-through? It remains to be seen. However, if you haven’t drunk the Kool-aid of using breadcrumbs on your website, I would suggest that now might be a good time to experiment.

Tags for This Post: google serps, subtle change, google, backlinks, kool aid, user experience

Web Page Load Time can Positively Influence Rankings

November 13th, 2009

Long Post Alert

This turned into a rather long post, so it’s probably a good thing I did it on a Friday.
However, this is a very important issue, so set aside some time to read through the whole thing.

I was reading a summary of topics that were presented at PubCon written by Rand Fishkin over at SEOMOz. One in particular caught my eye as I have been giving this very advice to clients for most of 2009: “Web Page Load Time can Positively Influence Rankings.”

It makes sense, doesn’t it? Google’s success is based on focusing on three key areas: user experience, monetization, and branding. Often, these principles may seem to be at odds, but it is the struggle that produces innovation. A tripod couldn’t stand if the legs weren’t pushing against each other right?

In this case, Google is focusing on user experience to feed branding, and may be sacrificing short-term revenue. In this case they may giving less focus to websites showing their content network ads and greater focus to established businesses that can afford to concentrate on optimizing load times.

The Most Important Issue in Load Times

One of my favorite anecdotes from my web career comes from a few years back when I was interviewing for a graphic designer position. A well-meaning fellow came to us with experience in ColdFusion and PaintShopPro (we were a .Net and Illustrator department at the time).

After running through our list of general questions we started asking the more existential ones, like “What is the most challenging thing you’ve ever had to do for a client?” The answer we got is hilarious to me, even today.

After a short explanation that “the web” ran on connections between machines, he launched into a short dissertation on the hex-code color system. Eventually we learned that, in his view, optimizing load times was best accomplished by taking a color which was used throughout the website (say “purple”) and gently dialing the hex codes closer to white, each time saving himself a bit of bandwidth usage (in theory).

Not only does this not make any sense from a design standpoint, it’s also fundamentally flawed from a tech-standpoint as well. But it does illustrate the desperation which plagues some designers in reducing their memory footprint.

Unfortunately, in the modern search landscape, kb-size means less and less. When you consider the cost of bandwidth (especially from Google’s perspective, who is amassing one of the largest fiber-optics collections in the world) the difference between downloading a 500kb page and a 5kb page is so small that it doesn’t matter, let alone the difference between 20kb and 18kb.

The most notorious factor now affecting this aspect of search rankings is one level deeper: it’s your server.

An Example of Server Response vs. Rankings

In 2008 I acquired a domain which had been abandoned by its previous owner. I set up a blog on that domain which stuck with the topics of the previous owner, and put in place a system which gave me 100s of pages of new content each week. By the end of 2008 the site was approximately 6,000 pages large, but with only a couple hundred pages actually indexed by Google.

During that time, the site was hosted on a shared platform. With approximately 3,000 other websites on the server, the load would sometimes spike to levels where the site was inaccessible, at least by the standards of the modern user.

As You Can See…

It’s almost a misnomer to say that your search rankings are positively influenced by load times. If anything, your rankings are negatively influenced, and anyone who doesn’t optimize their webserver is leaving money on the table.

I don’t have the exact numbers, but I would guess from experience that the site probably only had an 85% uptime (determined by pinging the site in 15 minute intervals). In March of 2009, I moved the site to a dedicated server.

The Not-so-Instantaneous Effects

When the site was re-launched on the dedicated server, I noticed an uptick in the number of pages indexed for the rest of March. Looking back, I would guess that these pages were considered “questionable” by the GoogleBot due to their unpredictable load times, and were suddenly being served very close to 100% of the time.

For the next month or so, nothing much happened. I believe Google was giving the site some time, the change in IP address probably tripped something on their end to wait and see what else happened.

Around May the site picked up momentum in leaps and bounds. In the space of a month I watched the number of pages indexed at Google jump from just under 1,000 all the way to the low 6,000s by the end of June. The site now stands at just over 14,000 pages indexed, which is roughly 75% of the total pages available.

Traffic similarly skyrocketed, from around 12,000 pageviews early in the year to a record 182,000 pageviews in August.

Caveats

Of course, it’s difficult to isolate the compounding variables in any SEO experiment, especially when you don’t know you’re experimenting in the first place.

There are a number of other factors that could have influenced the growth of this particular site: the fact that it now has a dedicated IP, or, since it is a blog, additional links in to popular posts. But I believe the predominant factor is the increased uptime.

Checking Server Response, and Why Load Time isn’t the Whole Game

There are many services out there that can help you monitor server performance, one of the most famous is PingDom. I’ve also used BasicState, which is a free service that sends you alerts and summaries when your server is unreachable.

It’s important to see how often your server is unreachable, not just how long it takes to load a page. You have to remember that for all the collective intelligence at Google, the menial tasks are still performed by robots.

Let’s say your site has 1,000 links coming in to it from other sites. And let’s pretend Googlebots follow those links at a rate of 10 per day, and that any given Googlebot will wait 10 seconds for your page to load before reporting that the site is gone and moving on. For simplicity’s sake, let’s also assume that all Googlebots communicate back to an Aggregator of some kind which stores data about your website.

On Day One, 10 Googlebots come by (or the same Googlebot 10 times, however you prefer to think of it). Your server is only capable of responding 8/10 times. Maybe one time it didn’t respond within 10 seconds, and one time it was actually down.

Remember, Googlebot is stupid. In the first case, Googlebot calls Aggregator and says, “The site is slow,”. In the second case, where the site is giving him a 404 error, he reads it as “Not Found” and decides the page is gone. He calls Aggregator and says “I got to this page via a link from someothersite.com and the page is gone.”

Aggregator sits back and watches this happen for 10 days. At the end of that cycle, he finds that Googlebot has reported 100 broken links (because the site was down when it got there) and 10% of the time the page did not load within the allotted time.

Aggregator compares this information to the limits set by Google Engineers, and data collected from other websites of this type. He then removes 100 links from your site’s profile (they go to “Not found” pages right?), and then places the site further down in the results, beneath websites that are better at responding to users.

The Bottom Line

The bottom line is that your website is an investment. If it’s worth it for you to spend time and money on an SEO strategy, then it’s worth your time and money to make sure you’re running on a strong server.

All too often, I see people get started by having their nephew build them a website. Eventually they realize they need to upgrade and hire a competent web-dev. Next they know they want more people on the page so they hire an SEO, who tweaks titles and builds links.

Nowhere in that entire chain did anyone think to upgrade the server. Servers are thought to be an issue for IT and Security, not Marketing. But in the world of Web Marketing, the underlying structure is just as important as the message.

To put it another way: would you print your next direct mail piece on the inkjet in the office? Or have it printed in full color on glossy cardstock?

Tags for This Post: Post, Alert, Long, response time, server, google, google results, search engine robots, seo

How Age Verification Strangles Your Website

September 24th, 2009

579005_breakthroughI find myself explaining this quite a bit since working with Coors and Co. and Miller Brewing Shop on their age-verification methodology, so I thought I’d do a blog post to answer some of the common questions surrounding this topic, which is becoming a major roadblock for alcohol, tobacco, and other age-restricted producers on the web.

Not surprisingly, the porn industry figured this out about 10 years ago.

What is Required of Age Verification and Why?

The beer brewing industry, like any smart industry of a certain age, has decided to voluntarily police itself, rather than risk government intervention in keeping minors away from their marketing materials, which can be costly, painful, and ultimately hurts business.

To that end, the Beer Institute has published some guidelines for brewers to follow, one of which is to ask people for their age before they can enter the site. Most brewers will also check information against a public records database before sending you a catalog anyway, so this is more a proactive measure.

So, the reason you need to fill in your age on every beer site is because breweries would rather police themselves than be policed by M.A.D.D. and the government.

Can We Help You?

If you’re looking for help in solving a problem involving age verification and search engines, chances are we can help. Feel free to contact us for a no-cost, no-obligation consultation about your specific challenge.

How Does Age Verification Interfere with Search Marketing?

Most breweries want to provide a branded page where they can verify a user’s age, and then deliver underage visitors to a page talking about their commitment to keeping alcohol away from minors (once again, to show that they are policing themselves).

For the typical developer looking at this requirement, this is a simple redirection to a form page and then a decision based on subtraction to another page. The craftier dev. will also include logic to forward the verified user back to the page they were originally trying to visit.

However, this poses a problem for search engine robots, who cannot verify their age (most of them are younger than 21, anyway). All they ever see is the age verification page, never the products. This results in them never showing the brewer’s site for no-brainer product searches like “coors mug” or “zima hat.”

You may think this isn’t such a big deal, but let’s take a look at a current example.

Last month, 110 people searched for “bushmills shirt.” That’s 110 people looking for a specific product. A Bushmill’s owned site is nowhere to be found in the search results, but there is a perfectly good selection of Bushmills’ shirts right here. You’ll have to verify your age to see that link, and remember they’re in the UK, so it’s day first, then month (that’s another post for another day).

Checking Google’s cache of that page reveals the problem. Bushmills is redirecting un-verified visitors to their age verification page.

Bushmills is missing out on anywhere from 1-110 t-shirt sales each month. And there are other searches as well, “bushmills hat” “bushmills glass” etc.

I have a list of over 100 alcohol brands. Of those, only a handful do age verification properly (literally 4 or 5). Some of them have blogs, images, and recipes locked up behind age-verification redirects. How much would it help a vodka brand to come up for a search of “Red Bull and Vodka Recipe” (bad example, who doesn’t know how that works?).

Incidentally, if anyone from DiaGeo is reading this, I would like to talk to you about helping with the Age-Verification on some of your apparel stores.

Tags for This Post: miller brewing, porn industry, search engine robots, proactive measure, verification methodology, age verification, alcohol tobacco

Click-Through on Organic Results

October 22nd, 2008

I spent some time this morning responding to an email on this topic, so I thought I’d post it here too as a reference.

There are three studies that I know of that have attempted to answer or predict the CTR of organic results. Each has a different methodology and sample data, but the results are similar.

First up, this one is based on the AOL data that was leaked a couple years ago. Someone plugged all the click data into a formula and got these results. Note, that this is AOL data, and it’s widely believed that those searchers behave differently than more savvy Google or Yahoo users: AOL Clickthrough Study Data

Second, this study was done by Cornell University using eye and click tracking equipment. It’s probably the most experimentally sound, but of course the people also knew they were in an experiment and were given specific search terms like “shopping” that might not be relevant to their state of mind. Cornell Organic Click-through Study

Finally, there is a third study done by me in my former life as a corporate SEO, which pre-dates the first two studies I mentioned. This one used keyword volume estimates and # of visits from a given keyword in a given ranking to a specific website. Because the dataset was smaller, I extrapolated the results to a logarithmic function which I have since forgotten. But the spreadsheet still contains the ranks.

The beauty of these results is twofold: 1. I think it better represents the B2B searcher and, 2. because it’s a function you can extrapolate the CTR on any given ranking (although since I don’t remember the formula, this sheet only goes back to result #200, but if you’re ranking past 20 you shouldn’t care about CTR anyway). Here’s a sample of my results, and you can download the full spreadsheet here:

Rank CTR
1 42.30%
2 11.92%
3 8.44%
4 6.03%
5 4.86%
6 3.99%
7 3.37%
8 2.98%
9 2.83%
10 2.97%

Tags for This Post: yahoo users, volume estimates, logarithmic function, cornell university, google

Directory Submission: An Inadvertent Case Study

April 18th, 2008

Directory SubmissionThe arguments against bulk submission of your website to hundreds of general directories can be summarized in the following way:

  1. Search engines will frown upon you getting 1,000 new links in one day.
  2. The value of 1,000 directory links from general-topic sites is next to nothing.
  3. Topical directories that are hand-edited will provide much more link-value, so you should spend resources on that instead.

I’d like to use an opportunity (or “crisi-tunity” if you’re a Simpsons fan) that’s recently come up for me to refute some of these claims. But first, let me refute #1 right away…

Rate of Change
When search engine spiders visit your webpage, there is a certain process that occurs. The spider reads the HTML of the page, probably makes a few statistical notes (# of words, # of links (internal and external), URL parameters used, dates on the page, etc.) and then puts it in the queue for analysis by a heavier-duty piece of software.

Your site gets put in the queue to be spidered in a few different ways: someone with a search-engine toolbar visits your site, someone links to your site, someone does a search for your specific URL. All of which give the site a different priority, depending on whether or not it’s in the index. Statistically speaking, this means that your site and my site are almost never being spidered simultaneously.

When you add in the fact that your page can’t be indexed until it goes through the indexing algorithSimpsons fanm (the heavy-duty software I referenced above) which can take a couple days, there is just no way for your new pages and mine to be added to the index at the same time.

Because links are an element of the page, they are “counted” in the same manner. When you spread this example across 1,000 sites, all with different indexing periods and rates there is just no way for a search engine to “see” you receiving hundred of links at the same time. They will appear to trickle in over the course of several weeks or months, especially when you consider the fact that not all the approvals go through at once either. I run several directories, and I only approve submissions once or twice a week.

There, #1 is officially dead, no?

The meat and potatoes
Last year a friend of mine was designing a brand new website for a company that sold promotional items. They were starting off with a brand new domain and wanted to get search engine traffic right away. Although my buddy asked me if I would help out, at the time I was confined by an agreement to only perform SEO for one company. So instead I hooked him up with a few tips on on-page optimization and a vendor who does directory submission.

Long story short the company ended up bailing on my friend after he delivered the site. They never paid him, so all that happened was the new site was built and live with on-page SEO basics, and it was submitted to 1,000 general-topic directories. One year later, the company has basically dissolved, but their site still stands (along with some analytics code I told my buddy to put on the site). Here’s what has materialized:

  • The site now has a pagerank of 4 (from N/A at the start).
  • The site shows 553 backlinks in Yahoo.
  • The site is receiving about 20 visitors a day from Google.

So it seems that those links are neither worthless nor ignored. They have produced tangible results in 2 major search engines. Granted, this won’t make you a million dollars, but for a $50 “fire-and-forget” submission package, why wouldn’t you?

What I Advocate and Response to #3
With regard to topical web directories and finding the really strong ones to submit to, I fully encourage you to take advantage. There will never, ever be anything wrong with achieving a high-quality link, from a site that is topically-related to yours.

However, if you have a brand new site where you’re just trying to get some “air under the wings” (or you’re running an Advanced Domaining Strategy) why not spend a little cash to get things moving? You can (and should) always supplement this with topical link-building, but that kind of strategy means you need to know your business, which is something you (conceivably) can’t outsource.

So spend your time building quality links, and outsource the foundational stuff.

Tags for This Post: , link building, directories, directory, seo

iSnare for the long term?

November 21st, 2006

The Article Distribution Service iSnare.com has been billed as one of the best tools around to increase a website’s presence. And I’ve been a big proponent of it since I first came across the service.

The idea is simple enough: submit an article to this service, it is reviewed by humans for quality and then gets auto-distributed to 1000s of article-aggregation websites, many on general topics, and a few on whatever topic you choose for your article.

After using it a few times, I began to notice that pages I promoted with the service would tend to rise in Google’s SERPs for my targeted terms, and then slowly fall back down. They would usually settle at higher positions than where they started, but I wondered why the Rome effect was so strong (that was a subtle reference to a rise/fall timeline).

So, I decided to study the Google results on fresh articles, and their mentions in search engines. I used the old trick of searching a unique phrase. On August 4th I used a unique phrase from each article on Google’s engine: 0 results. I then submitted both articles to iSnare for distribution. On August 8th I got an email that both articles had been approved and syndicated; a second Google search revealed 0 results for both.

0 results again on Aug. 9th. Then on Aug. 10th I saw the first signs of life: 7 results for Article 1 and 8 results for Article 2. By Aug. 15th, Article 1 had 437 results, and Article 2 had 458 results. There are two points of note here:

Point 1: I submitted both articles under the same category. They were approximately the same length (around 450 words). I submitted them on the same day within minutes of each other, and yet Article 1 lagged behind Article 2 for some reason.

Point 2: At this point (Aug. 15th) there were no supplemental results for either article. All 400+ results were fully viewable in the main index.

On the 16th of August the dupe filter must have kicked in on Article 1, because supplementals appeared and total results dropped to 361. Article 2 continued to thrive with 556 results on the 16th, with still no supplementals showing.

Eventually the dupe filter must’ve kicked in on Article 2 as well, and by August 30th, both result counts were below 50 (39 and 34 for 1 & 2, respectively).

As of today, Big G shows 11 results, of a total of 16 for Article 1 (so, approx. 4 supplementals). Article 2 fared better in the end, today displaying 16 results of 22 total (so, approx. 6 supplementals).

The [recently exported] PageRank for the top 10 results on each article range from 0-2, with the majority being 0 (and 2 N/As!).

So now some theories:

1. Article 1’s target phrase was more competitive than Article 2’s. My theory is that the more competitive an area, the greater number of filters (or in some cases, reviews) a page must pass to become part of the index. This is explained best in the theory of long-tail keywords, where phrases that don’t mean much in a marketing sense have a lot of impact on John Q. Searcher.

2. To compete with social bookmarking, Google needs to be buzz-aware. When a site creates a certain amount of buzz (linking, textual-references, etc.) Google needs to get in there and evaluate it for ranking. It will weight these sites with additional trustrank to get on top of the coming wave. A second (and potentially third) filter will later decide if the page is worth keeping in the index. Possibly by analyzing search volume for a phrase vs. the amount of “buzz”.

What might a takeaway be from this experiment? In my case, the combination of the “buzz” created with the article distro, plus the already-established authority (or Trustrank) of the site was enough to put the [brand new] pages I was targeting into the top 10 for their intended keyphrase.

As with most SEO activities, it is recommended to use this tool appropriately, and in combination with other tools.
Any thoughts?

Update: Looks like Aaron Wall and I may have been thinking along some similar lines.  He just posted about  new domains getting ranked in Google over old sites, and mentioned the following:

“Also think of the search business model as though you are a search engine. To them, being the first person to do something is a sign of quality because to be the first person in a market requires some market timing / knowledge / investment / luck.”

Tags for This Post: google results, signs of life, serps, subtle reference, google search, first signs

Are Meta tags still useful for SEO?

November 17th, 2006

I was reading a thread about meta tags on the DigitalPoint forums today.

.NET magazine wrote an article on SEO recently and stated that meta tags no longer matter for search engine optimization. Were they correct to do so?

The SEO industry is in state where most of the mechanical aspects of optimization can be handled by a competent web designer who stays on top of the basics (.htaccess, robots.txt, static URLs), which is why we see so many design firms now touting that they can perform SEO while designing a site.

So that takes care of the ’search engine’ part of SEO. But what about optimization?

Search engines (specifically Google) still use meta tags, but not to rank your site. At least not directly.

Let’s start with three statements:

Fact 1: Google has access to your meta- title and description tags.

Fact 2: Google will display your title and description tags in search results, unless the engine feels that writing its own description from your text, or using your DMOZ entry is more relevant (algorithmically determined, happens less than 20% of the time in my experience).

Fact 3: Google will rearrange its search results by click through rate and other (measurable) factors to provide he most relevant results to users.

You may wish to debate fact #3, but for now let’s assume it’s true in this world.

The acceptance of these three facts means that you need to write a title and description that will impress visitors and increase your click through rate. A better visitor experience will push your site higher in the rankings.

So how do you write a good title and description? Well, that means it’s time to take a page from the PPC-world…

Tags for This Post: relevant results, digitalpoint forums, description tags, mechanical aspects, search engine optimization, google, visitor experience, meta tags, design firms

Finalist at MarketingPilgrim.com

November 10th, 2006

So it turns out the article I submitted to the MarketingPilgrim SEO Scholarship Contest was chosen as a finalist. This means my article will also be judged by a panel of professionals for the grand prize.

I had hesitated to enter the contest, but I’m glad that I did. I wanted to go a different direction than the previous entries, and talk less about why SEO is important and general, and tackle a specific issue that I had noticed causing problems for people. As I say in the article, I talked about the same thing at an AdWords conference and I had the whole table listening intently.

Anyway, if you’re arriving here for the first time, please be aware that I’ve recently taken down much of the content on this site and I’m gradually replacing it with better, more informational articles. I decided that since the guys over at MP were good enough to give me a big fat link to this site, I’d better make it worthwhile to visit. Bookmark my site and stop by later.

Or you might see me again in future articles!

Update: So I tied for second in the final round. One of the judges had this to say of my work:

I’ve got to give it up for the direct, hands-on learning and image integration shown by Ryan Bell in Instant PPC Success.

Thanks for a fun contest guys!

Tags for This Post: thing, scholarship contest, grand prize, image integration, mp, Bookmark, informational articles

Laying the Groundwork for a Successful Site: Part 1

November 9th, 2006

You’ve just completed your site. You have a unique offering to the public, but how are you going to get the word out?

If you’re going to get the maximum benefit from your future SEO activity you need to lay the proper foundation. Step one in the process is getting the proper URL structure set up.

The first decision you need to make is to decide whether you want your site to appear as www.mysite.com or just mysite.com in search results. There isn’t necessarily a difference, and it’s a matter of personal choice.

One factor that might help you make your decision is that less computer-savvy folks might be more inclined to add the www. before your address, whereas “techie” types will be more likely to use just mysite.com.

If your website is hosted on an Apache server, you can add this code to your .htaccess file:

RewriteEngine On
RewriteCond %{HTTP_HOST}  !^www.example.net$       [NC]
RewriteRule ^(.*)         http://www.example.net/$1  [L,R]

For an IIS server (with isapi_rewrite), you can use the following code:

Add this to your httpd.ini file:

RewriteCond Host: ^mysite.com
RewriteRule (.*) http://www.mysite.com$1 [I,RP]

If you want to rewrite your URL to the non-www version your Apache code would like:

RewriteEngine On

RewriteCond %{HTTP_HOST} ^www.mysite.com [NC]
RewriteRule ^(.*)$ http://mysite.com/$1 [L,R=301]

Next up, creating the proper URL structure for your navigation.

Tags for This Post: www version, maximum benefit, savvy folks, techie types, rewriterule, www mysite com, url structure, proper foundation, proper url