Locating Broken Links With Xenu and Fixing Canonical Errors With Your Htaccess File

I’m currently working on a natural listing keyword campaign with Network Solutions and had to fix a number of Canonical errors. To track down the offending pages I ended up using a free program called Xenu. Since the web site I’m optimizing has over 1000 pages I needed to find a way of doing this more efficiently than hitting each page 1 at a time.

Download Xenu For Free Here

How To Use Xenu

What is a Canonical Error?

Once I ran Xenu I realized that the site I am optimizing had 100’s of Canonical errors on the web site we’re promoting. A Canonical error is the creation of duplicate content due to the way web server software handles variations on URL’s. See the example below.

http://www.yourwebsite.com < - Correct
http://yourwebsite.com <- Canonical error

Since we’re gearing up for a major campaign it’s imperative to eliminate what web site errors we can. Since these errors will be indexed on all the major search engines it’s in your best interest to fix what errors you can so your users don’t wind up on a broken page.

Both links above will usually take you to the correct page, but some search engines will punish you for duplicate content. In order to not get dropped it’s a good idea to remove these issues when you can.

What Is A Natural Listing?

There are 2 types of listings for the search engines. PPC (Pay Per Click) and Natural. A natural listing is a listing on a search engine that you don’t pay for. It is thought, and I to believe, that natural listings pay off better than PPC efforts. Statistically more people click on natural listings than on paid ones. I know it’s certainly the case for me.

Solve Canonical Errors With Your Htacess File

“An htaccess file is a simple ASCII file, such as you would create through a text editor like NotePad or SimpleText. Many people seem to have some confusion over the naming convention for the file, so let me get that out of the way.”

-From JavaScriptKit

Your .htaccess file tells the server how to process pages. To tell the server how to interpret your web page paste the code below into your htaccess file. .htaccess files must be uploaded in ASCII mode, not BINARY.

RewriteRule ^/index\.(php|html)$ http://www.yourdomain.com/ [R=301, L] RewriteRule ^(.*)/index\.(php|html)$ http://www.yourdomain.com/$1/ [R=301, L]

-WebProWorld

This stirp of code rewrites the offending URL and eliminates the problem. To double check yoru site simply run Xenu again and check for errors. I went from around 1000 to just under 20 by adding this simple strip of code.