Inside Google Sitemaps: More stats and analysis of robots.txt files
by Kent Lewis on February 16, 2006UncategorizedGoogle Sitemaps now shows you a list of the most common words in your site’s content and in external links to your site, which can help you build and refine your keyword lists to target your audience. It also includes the page on your site that has the highest PageRank, by month, for the last three months. Additionally, a robots.txt file analysis feature has been added that provides reports on how Googlebot views the file and allows users to test the robots.txt file before it’s implemented on the site.