Robots.txt is a text file which sits on the root of your domain and instructs search engine crawlers which page or directories of the site to crawl and which not to There are certain files in your blog like the admin folder which you might want to disallow the search engines from indexing. Blogs which […]


Read More...