Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Robots.txt Generator tool!

The robots exclusion general, also called the robots exclusion protocol or surely robots.txt, is a fashionable utilized by web sites to speak with net crawlers and other internet robots. the same old specifies how to inform the net robotic approximately which areas of the internet site ought to no longer be processed or scanned. Robots are often used by search engines like google to categorize net sites. now not all robots cooperate with the same old; electronic mail harvesters, spambots and malware robots that test for security vulnerabilities may additionally even start with the portions of the internet site where they were informed to live out. the same old isn't like, however may be used along side Sitemaps, a robot inclusion popular for websites.