How Do I Direct Spiders Around My Site?

hanging complete spider webs

That’s a good question as people new to site building aren’t aware that there are web spiders (bots), much less 3 types of site maps. Bot’s are essentially discovery programs used by the search engines. The bot’s roam around the web, finding new web sites and web pages by use of  common web links. Sitemaps are repositories of a web sites resources/pages, represented as links. Each kind of site map is similar but with it’s own specific purpose. With out sitemaps, bots would have difficulty finding all your site web pages and we all know disastrous that would be!

The three types of sitemaps used to direct the spider around your site are sitemap.html, sitemap.xml & sitemap.xml.gz. Way back during the ‘wildwest’ days of the internet (10 years ago), web surfers liked a concise way to scan the contents of the site they were viewing and, necessity being the mother of invention, the site map was born.  This type of site map was really only a normal ‘.html’ page containing short descriptions and links to the sites resources. Later on as the web matured (5 years ago), the site map was also used as a way for spiders to find all you sites pages. The only real reason for this was due to poor or broken site navigation.

Now that we have entered ‘Web 2.0′, the site map has been reincarnated as a search engine specific tool. This current not for human consumption incarnation of the sitemap is designed to let the search engine spiders find each page of our web site with out having to follow hyperlinks throughout the site. This type of sitemap uses the ‘.xml’ format file extension and is located at the root of the publicly accessible portion of the domain. The third type of sitemap is a Google specific archive ending in .xml.gz and is simply a compressed version of your .xml sitemap.

Just in case you are wondering, each file will contain the same destination pages, the only difference is in the format in which the information is delivered. Ideally speaking, every site really should have each type of site map present and a reference in the robots.txt.

There are many brands of software available to create sitemaps of each type. When using a Worpress blog, they are especially easy to create with simple plugins.

So there you have it!

Related SEM-Answers Posts

• Article Writing 2.0 Maximize Your Article Marketing Writing for and marketing with web 2.0's New Media for Virtual Real Estate Naturally internet marketers know that if you want to drive quality traffic to your site what you really need is quality information creating the push. That is the way marketing has been done......
• How To Make Money Online Series How To Get Started Making Money Online Series Article 20 OK so if you’re going to make money online, in most cases, aside from getting an email address, the first thing you are going to need is a place to publish your content. You can publish for free or you......

Tags: html sitemap, xml sitemap, xml.gz sitemap

Stumble It

Ping this!