CUSTOM ROBOTS HEADER TAGS AND PURPOSE
1. all – If you set this tag, crawlers are not bound by any constraints. They can freely crawl, index and expose your content.
2. noindex – Not all the blogs are for public notice. Even if you don’t share the URL of your personal blog with anybody, chances are people will come to it from search results. On such a scenario, you can use noindex tag as it prevents search engines from indexing the pages.
3. nofollow – Nofollow and dofollow tags are for outbound links. Dofollow is the default robot tag for all your outbound links. That means the search engines can sneak upon the pages you linked to. If you don’t want search bots to look through your links, addition of a nofollow tag must help you.
4. none – none combines the features of both noindex and nofollow tags. The crawlers will neither index your pages nor skim through the links.
5. noarchive – You might have noticed a cached label with most of the website links on SERPs. It shows that Google has captured a copy of your site into their server to display in case it goes down. That being said, the noarchive tag turns off cached version in search pages.
6. nosnippet – The text snippets in search results help people find what’s on the webpage. If you want to keep the content exclusive, you can turn this header tag on.
7. noodp – Open Directory Project or Dmoz is a man-made directory of websites.Google use the information from there sometimes. You can turn it off with this tag if you want to.
8. notranslate – Do you want to disable translation on your site? Then use notranslate for the exact purpose.
9. noimageindex – If you allow Google to index your images, people may steal it and use on their own websites. To prevent that, you can keep the images deindexed using noimageindex tag. Also Read this Image SEO guide.
10. unavailable_after – In Blogger, you will get a field right to this tag. So, the webpage will be deindexed after this time.
Leave a Comment