Spiders like Googlebot choke on Session IDs
Many ecommerce sites have session IDs or user IDs in the URL of their pages. This tends to cause either the pages to not get indexed by search engines like Google, or to cause the pages to get included many times over and over, clogging up the index with duplicates (this phenonemon is called a “spider trap”). Furthermore, having all these duplicates in the index causes the site’s importance score, known as PageRank, to be spread out across all these duplicates (this phenonemon is called “PageRank dilution”).
Ironically, Googlebot regularly gets caught in a spider trap while spidering one of its own sites – the Google Store (where they sell branded caps, shirts, umbrellas, etc.). The URLs of the store are not very search engine friendly: they and are overly complex, and include session IDs. This has resulted in 3,440 duplicate copies of the Accessories page and 3,420 copies of the Office page, for example.
If you have a dynamic, database-driven website and you want to avoid your own site becoming a spider trap, you’ll need to keep your URLs simple. Try to avoid having any ?, &, or = characters in the URLs. And try to keep the number of “parameters” to a minimum. With URLs and search engine friendliness, less is more.
Possible Related Posts
Posted by stephan of stephan on 06/25/2004
Permalink | | 
Print
| Trackback | Comments Off on Spiders like Googlebot choke on Session IDs | Comments RSS
Filed under: HTML Optimization, Spiders
Google’s Gmail servers down
I guess it can happen to the best of us.
Possible Related Posts
Posted by stephan of stephan on 06/20/2004
Permalink | | Print
| Trackback | Comments Off on Google’s Gmail servers down | Comments RSS
Filed under: Google
On Orkut, I feel like a voyeur
This weekend I finally got an invite to Orkut, the exclusive closed social network site developed and run by Google. I feel like a voyeur, inconspicuously ascertaining who’s friends with various mega millionaires like Sergey Brin and Steve Jurvetson. It’s rather surprising what some business people reveal in their profiles — stuff you certainly wouldn’t reveal to your own mother. Don’t they realize people actually read these things? Anyways, Orkut was great fun for a while but now I’m already getting bored with it.
Possible Related Posts
Posted by stephan of stephan on 06/12/2004
Permalink | | Print
| Trackback | Comments Off on On Orkut, I feel like a voyeur | Comments RSS
Filed under: General
