Free Articles  >>  SEOptimization >>  Page 90  >> 

Google Wherefore Art Thou Google Sites Abandoned by Googlebot

Google, Wherefore Art Thou Google? Sites Abandoned by Googlebot!   by Mike Banks Valentine

Google, Wherefore Art Thou Google? Sites Abandoned by Googlebot!

© August 30, 2004

As a search engine optimization specialist I often optimize
existing web pages for small business clients, upload them to
the site and see pages re-indexed by Google within a week.
This only happens with existing business sites that have been
online for a few years. Google seems to be updating their
index as often as every other week at this point and older
established sites that are already indexed seem to be re-
crawled on that twice a month schedule on a fairly routine

Two clients that hired me for recent work saw their rankings
shoot to the top for a newly targeted search phrase in a
weekend when I did optimization on a Thursday and they were
ranked instantly by Saturday. Now keep in mind that this
doesn't happen for everyone, only those that have been online
for some period and already have significant content that
simply needs tweaking and proper title and metatag information
added. They usually have relatively good existing PageRank and
do well for other RELEVANT search phrases already. I offer that
warning only to avoid instilling false hopes in anyone hoping
to achieve the same instant ranking boost overnight.

Those clients that do succeed in this way are often thrilled
with the results accomplished in such short order. I'd love
to be able to offer that type of ranking boosts to everyone,
but some are more equal than others when it comes to easy,
inexpensive SEO tune-ups that rev up your rankings overnight.
Your mileage may vary.


What is going on with newer sites that don't get crawled for
months? I've got a client, a newer attorney directory that
offers tons of great information in the form of articles on
specific areas of law, links to incredibly valuable and
relevant legal sites and over 600,000 attorneys listed by
practice area and state. Yet the site has not been re-crawled
by Google for over 3 months! Now this would not be such a big
issue for many sites, but this site is relatively new and we've
optimized all the titles, tags & page text, created a complete
site map and placed links to all these resources on the front

I know that the site is not being crawled because Google's
cached copy of the front page shows it before we did the
work four months ago, without the new links and without
title tags. We've submitted the site by hand, (manually)
once a month for three months via the Google Add URL page. When the hand submission
failed to get it re-indexed for four months, we submitted
the sitemap page, which has not been crawled at all. Google
shows only ONE page on this site, when in fact it has
thousands of pages, a sitemap and dozens static pages!

Part of the problem is that this site must be dynamic, since
a database of over 632,000 attorneys must be accessed,
retrieved and served for any of those law firms searched for
to be returned to the site visitor. Google warns owners of
dynamic sites that Googlebot may not crawl dynamically
generated pages with "?"" question marks in the URL. This is
to avoid crashing the server with too many concurrent page
requests from Google's spider.

The solution to this dynamic URL problem has been discussed
widely in search engine forums and solutions have been bandied
about including software provided by SEO's, URL re-write
techniques for dynamic pages on APACHE servers and PHP pages to generate search engine
friendly URL's. Others recommend simply adding static HTML
sitemap pages as alternatives for the search engine spiders.

In this instance the client's developer simply said "I
do that (PHP solution) on this server". So we resorted to
putting up the static HTML sitemap pages with hard-coded
URLS to the main 54 pages of the site at This should
get at least those fifty pages crawled by Googlebot, but
Googles' spider appears not to be crawling this site at all.

How do we know this? See for yourself by using the following
query in the search box at Google:
where the result page shows ONE page in the results. If you
try that query on your own site (replace your own domain name
for, you'll see the results lists ALL your

The site home page was crawled by Google four months ago, when
they took their "Cached Snapshot" of the page. You can see
this by visiting the Google cached page here:
where the date of this snapshot is "Apr 20, 2004 07:42:19 GMT"
and they haven't been back since. The page in that snapshot
has none of the newly added links, an outdated title tag, and
old content.

This problem is not unique to this site. One client we worked
with two years ago had a dynamically generated, framed site!
Those two site structures have always given search engines
trouble. Their site was not crawled at all and only the front
page showed up. Our solution was to create a second domain
(owned by the client), which had static HTML pages that
precisely mirrored the content of the client's framed,
dynamically generated site. Guess what happened after
Googlebot crawled the static site? Google indexed the framed
site in full and then banned the static site from the index!

Not an approach we advocate, but the one that worked for this

We're still searching for ways to get Googlebot back to before creating that new static site, but
decided to share this odd experience with the SEO community
before going to any extremes. Google provides over 70% of
most search engine referred traffic to ALL of our clients
and we realized we can't site idly by and see a major client
languish because Googlebot didn't like what it found at the
client site on the first visit four months ago.

This issue dogs newer sites in other places as well. The Open
Directory Project has also become notoriously slow in adding
new sites to the directory and in this case, has not picked
up this site even after 6 regular monthly submissions. The
web playing field may have begun tilting toward older,
established sites and away from new ones.

About Author Mike Banks Valentine :

------------------------------------------------------------Mike Banks Valentine is the SEO for http://www.lawfirm411.comContact him at Improve Your Small Business Online at our Ecommerce Tutorial

Article Source:
Article Url:

Other Articles by Mike Banks Valentine

New MSN Search May Be a Google Killer
   by Mike Banks ValentineThe Second Look at MSN's Search technology is available forpublic beta testing. I've given it a spin myself and must saythat I'm impressed. Although they have no ads on the SERP'sof the preview site, I'm sure they will load it up with the15 or more Overture sponsored sites which clutter the resultspages on the current search results by the time the new publicbeta reaches its official launch. Take a look for yourself:'ve publicly...

Google Tops Fifty Billion Market Cap With Innovation
   by Mike Banks ValentineYahoo See, Yahoo Buy Strategy Makes Them Number Twoby Mike Banks Valentine © October 26, 2004Google market value has topped $50B, so now they're worth more in dollar value than Yahoo. That goes right in linewith being worth more in search value and relevance. (CBS MarketWatch)I've written before that Yahoo has their fingers in so manypies that they don't have the necessary attention on searchthat Google does. Google only buys companies that offerservices, software...

Google Spyware Bad Guys amp Spies Using Google Desktop Search
Google Spyware? Bad Guys & Spies Using Google Desktop Search   by Mike Banks ValentineI suppose I was naive when I cheered the new Google DesktopSearch tool thinking it was ONLY a great way to help ease mycomputer info-glut and help organize my hundreds of hard-drivestored documents, emails and files. It seems that now I haveto worry about how bad guys and busybodies will use it tospy on me! ( Google Desktop Search Tool Poses a Security Risk tousers of public or networked...

Google Bombing or Legitimate SEO
?   by Mike Banks Valentine Today I visited the WebProWorld forums and stumbled acrossa topic New Google Bombing Technique and was startled tosee that the topic had been viewed over 22,000 times! It isby far the most active of the forums in the Search EngineInsider Reports section.This article is simply my response to that discussion threadand I hope it helps cool off the heat being generated there.Google Bombing DiscussionThe general gist of the conversation suggests that appendinga question...

Google Desktop Search versus Microsoft Windows Search
by by Mike Banks Valentine © October 17, 2004or Honey! Have you Seen My Keys, Glasses, Tivo Remote?Google Desktop Search Software can't find your lost keys ortell you where you left the Tivo remote control, or that yourglasses are on top of your head, where you left them. But thebeta software from Google Labs is nothing short of mandatoryfor those with more emails, Word documents, Powerpoint, Exceland PDF files than they know what to do with. That's me.New fixtures in our lives can become...

Throw Away Your Money on Search Engine Optimization
   by Mike Banks ValentineSearch engine optimization consists of some relatively arcaneissues that are not obvious to anyone. That I can be thankfulfor, I suppose, or I wouldn't continue to be in demand as anSEO specialist. But why do clients throw money out the windowwith developers who don't understand they are bulldozing downsite naming structure and careful page architecture when theydo a site redesign?Today I got a call from a very good client who was excited tohave me see his site...

To Be or Not To Be That is the Domain Question
by Mike Banks ValentineAdaptation by Mike Banks Valentine with apologies to ShakespearTo be, or not to be: that is Domain question.Whether 'tis nobler in the mind to suffer The slings and arrows of slow, free servers,Or to take arms against a sea of wannabees,And by opposing, end them. To search: to seek;No more; and, by anonymity to say we endThe heartache of a million useless search resultsThat eBiz is heir to, 'tis a relevancyDevoutly to be earn'd. To look; to seek;To find? perchance to...

Affiliate Programs Worthless for 80 of Affiliate Marketers
by Mike Banks ValentineNo doubt you’ve heard of the 80/20 rule – which states that20 percent of affiliates generate 80% of affiliate income, andconversely, remaining 80 percent of affiliates generate only20 percent of all income for most affiliate programs.There are ten basics that will put your affiliate marketingin the top 20 percent of affiliate earnings. Ten suggestionsthat will transform your affiliate income from a tricklingcreek that barely covers your hosting fees, to a roaring...

Shopping Cart Abandonment Solved
by Mike Banks ValentineImagine yourself at the local Sears carrying a shopping basketdown the isles looking for the perfect gift for your spousewhen you come to a great looking watch in the jewelry department.You know she'll love the design, but also know she has a curiousallergy to white gold and call over a salesperson to ask themwhat the bevel surround is made of. She reassures you that it'smade of platinum so you cheerfully purchase the watch, smilingas you wander over to have it...

1 800 Get Rich Can Toll Free 800 Number Domains Pay Off
?  by: Mike Banks Valentine The toll free number 1-800-Get-Rich belongs to the Resorts Casino Hotel in Atlantic City. Perfect vanity number for a casino, right? Well apparently not. Their website shows the actual numbers, 1-800-438-7424 for the marketing department of Resorts Atlantic City. Those NUMBERS are nowhere near as memorable as is the mnemonic device of letters representing those numbers on the telephone keypad. It makes you wonder, did the casino have bad luck (no pun intended) or...

Click here to see More Articles by Mike Banks Valentine
Publishers / Webmasters
Article ID: 22990
DELINK URL from Authors Bio
REMOVE Article
Tell A Friend
Leave A Comment!
Download this article in PDF
Report Article!
Search through all the articles:

246 Users Online!!
Related Articles:
Latest Articles:
SEOptimization >> Top 50 Articles on SEOptimization
Category - >
Advertising Advice Affiliate Programs Automobiles
Be Your Own Mentor Careers Communication Consumers
CopyWriting Crime Domain Names DoT com Entrepreneur Corner
Ebooks Ecommerce Education Email
Entertainment Environment Family Finance And Business
Food & Drink Gardening Health & Fitness Hobbies
Home Business Home Improvement Humour House Holds
Internet And Computers Kiddos and Teens Legal Matters Mail Order
Management Marketing Marriage MetaPhysical
Motivational MultiMedia Multi Level Marketing NewsLetters
Pets Psychology Religion Parenting
Politics Sales Science Search Engine Optimization
Site Promotion Sports Technology Travel
Web Development Web Hosting WeightLoss Women's Corner
Writing Miscellaneous Articles Real Estate Arts And Crafts

Disclaimer: The information presented and opinions expressed in the articles are those of the authors
and do not necessarily represent the views of and/or its owners.

Copyright © AwareINDIA. All rights reserved || Privacy Policy || Terms Of Use || Author Guidelines || Free Articles
FAQs Link To Us || Submit An Article || Free Downloads|| Contact Us || Site Map  || Advertise with Us ||
Click here for Special webhosting packages for visitors of this website only!
Vastu Shastra

Linux Web Hosting In India Provided By AwareIndia

Company IDS