Adding Custom robots.txt in Blogger 2016

Now in Modern Days, Many new bloggers who create new blogs but don't know how they set-up their blog but doing some settings. They posting on regular basis but no idea how people reach his or her blog. Or their blog friends told him or her to edit custom robots.txt to get and reach traffic. You know after Submit your website to google search engines or yahoo/bing search engines You have to edit custom robots.txt in order to your website index by spider and bots. You are come to right place in this tutorials we are going to touch Adding custom robots.txt in blogger 2016.

Before going to add custom robots.txt we have to understand what is this and why is so important to our blog. There is also a Warning: Incorrect use of these features can result your blog is being ignored by search engines.

Custom Robots.txt:

Every Blog who post a content on their website or blog, An robots.txt file is auto generated. This file is generated because the reason is to inform spiders, bots or crawler, sent by google search engines and bing or yahoo search engines about your blog that new content is posted and they come to index and crawl your website to appear in search engine. Its your depends upon you rather you allow crawler to crawl and index your site. Being a blogger I think every blogger wants to see its blog is becoming popular, While others not to prefer index page like demo pages, Private page, Label pages, or irrelevant pages.

Where is robots.txt in Blogger:

Well login to your blogger select your blog and go to dashboard. Now Go to Settings >> Move to Search Preferences and There is option crawler and indexing below this option custom robots.txt is placed. Or Other option you can simply go to http://yourblogadderess.blogspot.com/robots.txt , See the below picture to clearly understand what am i taking about:

How Robots See your New Content:

Robots well its is spider they see your content by coming to your website. Your Custom robot.txt is like an security in plane where it see that you are going to right direction and don't enter to private areas.Up coming content can only be indexed that robots.txt would tell to up coming spider all other content saved from indexing or crawling.

Your Custom Robots.txt:

Well if you not touched to your custom robots.txt then it is look like this:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://myblog.blogspot.com/feeds/posts/default?orderby=UPDATED

Don't worry if your custom robot.txt have no color or line breaks. These added by me because you can clearly understand what is that code.

Understanding Code:

User-agent: Mediapartners-Google:

Google-Media partners is a google adsense program robot that would often visit your site and checks the rank of ads server on your website or blog. If we remove this then adsense program robot would not be able to see ads. Similarly if you are not using google adsense program just remove it. Remember if you not using adsense if you using it. Then this is important otherwise your account may be block and you would not be able to server ads again on your website.

User-agent: *

If you have an programming experience than you guess that this upcoming * sing is wildcard. For those who have not any kind of experience this portion describes that those line beneath is for upcoming spider, bots, crawler in order to index your content and server in there search engine.

Disallow: /search:

Disallow keyword means that "not to do" and then adding /search next to it describes that you telling bots not to crawl or index search page of your blog or your website. Example: An Example of this http://www.how2tutorials.blogspot.com/search/lable etc.

Allow: /

You probably know that what the meaning of allow in here its means that allow means "to do" its mean we enabling google or search engine bots to crawl specific page of your website or blog adding "/" means that your home, We are allowing google or search engines bots to crawl our website or blog homepage.

Sitemap:

Sitemap keywords refers bots to our website. Basically its tell the crawler that crawl or index every new post of blog or website. By specifying it with a links, we are now optimizing the crawler efficiency for incoming robots find the path to find entire links of our website or blog.

The Default Sitemap index only 25 post, If you are posting daily and have large number of post your probably need other sitemap.

Sitemap for those who have less than 25 post default:

Sitemap: http://myblog.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500

And for those who have large number of post more than 500 or 1000 use this sitemap:

Sitemap: http://myblog.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500

For 1000 Post:

Sitemap: http://myblog.blogspot.com/atom.xml?redirect=false&start-index=500&max-results=1000

Now I think I explain very much and you understand that what is custom robots.txt Now move to our main point how to add custom robots.txt on blogger:

Adding Custom Robots.txt:

Sign in to your blogger profile chose blog and come to dashboard
Now move to Settings>> Search Preferences >> Crawling and Indexing Option
Now go to custom robots.txt click Edit and click yes to enable custom robots.txt

4: Copy paste the robotx.txt code and make per changes according to your needs.

5: Once you are done click on save changes and boom. Now your content is crawl by bots and spider or search engines.

If you want to see if their is any changes in your custom robot.txt

got to www.yourblogadderess.blogspot.com/robots.txt

Now you understand the code and how to add this, if you not get it read it again i know its long but Make a learning your passion and its become fun for you. I keep updating you share it with your blog friends and help them to increase their site rank.