Must Read
To increase your blog performance, you need to optimize it for better SEO. A better SEO will automatically drive organic traffic from search engines to your blog. For Affiliate marketers, this means lots of leads to be generated. One great step towards a better SEO is adding robots.txt file to your blog. It tells the search engines crawlers about what page to crawl and what not to. In Blogger (Blogspot) we have the option to customize the robots.txt file according to our needs. But before adding it to your blog firstly as a beginner we need to know about what is robots.txt file. So lets start with the tutorial right now.
Robots.txt: is a file which tells the search engine crawlers what page to crawl and not to. Simply, you can exempt your blog demo pages, advertisement page, label pages, archive pages and other which you don't consider as important. Search engine crawlers always scan your robots.txt file before crawling your blog.
So, lets start adding this file to your blog to help you improve your blog SEO.
To add Custom Robots.txt file to your blog, follow the simple steps mentioned below:
1. Go to www.blogger.com and login.
2. Navigate to Settings >> Search Preferences ››
3. Crawlers and indexing ›› Custom robots.txt ›› Edit ›› Yes
4. Now paste the below robots.txt file code in the box to look like this
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://YourBlogUrlHere.com/sitemap.xml/?redirect=false&start-index=1&max-results=200
5. Remember to change the highlighted url (YourBlogUrlHere.com) with your blog address.
Disallow: /search in the above code is telling search engines not to crawl any of your blog page with /search.... attached to your URL e.g. https://www.givemereport.com/search/label/News
Before you click to save, please read the below explanation of the above code to understand what you are about to do.
Allow: / part of the above code is telling search engines to crawl all pages with the / sign attached, but without the word "search" attached to the above sign. Any other word can come after /, but definitely not the "search" word.
max-results=200 is telling search engines to index the recent 200 updated posts on your blog.
Are you ok with the code?
6. Click on Save Changes button.
You are all done.
Simply add /robots.txt after your blog address and you will be shown your robots.txt file.
I hope you have easily added the custom robots.txt file to your blog? Please click below to share this post with friends.
