Google is the largest search engine in the world today. And since it is the industry leader, it is only natural for it to set the standards....
Google is the largest search engine in the world today. And since it is the industry leader, it is only natural for it to set the standards. All bloggers and webmasters have only one concern in their minds regarding their sites when it comes to Google: whether Google is indexing their site or not? That is understandable, since most of their website traffic comes from Google. So to get un-indexed by Google is like having your legs chopped off - your blog won't be going anywhere, anytime soon. Well, to put some of the concerns regarding indexing at bay, Google has introduced a new Index Status feature in the Google Webmaster Tools. This feature will let webmasters know how many pages of theirs' have been indexed, crawled, and more!
Where can you find this information?
To check out this feature, log into your Google Webmaster Tools account. On the home page, click on the site for which you want to see the index report. Now click on the Health option on the left side of your dashboard, and select Index Status from the available options. Now, you will see two tabs, Basic and Advanced.
The Basic tab
The basic tab shows a graph which tells you how many pages of your site have been indexed over time. It shows up to one year of data. And then legend will tell you how many pages are currently being indexed. Google says this information is accurate, and does not include non-canonical URLs. So if there are more than one URLs pointing to the same content, or if there's duplicate content present, then it will be counted just one page instead of multiple pages.
Google also admits that this data lags by around a week. So this isn't a real-time reporting. Rather, it is better suited for monitoring trends. If the graph has been steadily going up, then there's nothing to worry about. It means that your new content is being recognized by Google, and is being indexed. If the graph remains stationary, or falls down despite your addition of new content, then you might want to look into the Advanced tab.
Advanced
The Advanced tab gives more insights into how Google has been treating your website. In addition to the Total Indexed graph, you can also see the Ever Crawled, Not Selected, and Blocked By Robots graphs.
- Ever Crawled - The total number of pages Google has ever come across on your site. This will show you your maximum total number of pages. So if you had 100 pages which Google crawled, and then you deleted half of them, this will still display a count of 100.
- Blocked By Robots - Number of pages blocked by robots.txt. Google knows about these pages, but wouldn't index them unless you change your robots.txt file
- Not Selected - Total number of pages that Google hasn't seen fit to be included in its search results. This could happen due to a variety of reasons
- It has a redirection to another page
- It has duplicate content also found on another page, and hence the other page was indexed instead
- It has a rel="canonical" tag to other pages.
How this data is useful?
This data could be really useful for debugging purposes to see what's wrong with your website and why you aren't getting traffic. An increase in the number of Total Indexed and Ever Crawled pages of course means that you site is on the right track, and that Google can easily access your content.
However, if there is a sudden drop, then it means Google is having trouble reaching your website. It could be because your web server is down, or that, you added a sitewide "noindex" tag rather than adding it on just one page.
A large number of Not Indexed means there are problems with canonicalization, or there's a lot of duplicate content on your website. You should make efforts to reduce the amount of duplicate content and non-canonical URLs on your website.
The Google Webmaster Tools is your friend. It will notify you of any errors on your site, so you can easily correct them. It is helpful features like this that make Google the best search engine on the planet!
COMMENTS