Interactively create a robots.txt file to indicate which robots you don't want crawling your site, and which files or directories you don't want crawled.

1 Choose your default robot access
Allow all robots (Recommended)
Block all robots
2 Specify any additional rules
Crawl-Delay
Sitemap
Example: http://robots.googletoad.com/sitemap.xml
(leave blank for none)
Action Robot Files or directories

If you want this rule to affect all files and directories, type /. To specify individual files or directories, type them one per line.
Example:
/directory/file.html
/directory/subdirectory/
3 Create your robots.txt file
4 Save your robots.txt file
To save your robots.txt file, click Download, or paste the text below into a .txt file. Name the file robots.txt, and save it to your site's top-level directory

 

2004 - 2023 www.googletoad.com

You can send an email to info@googletoad.com or call or text to webmaster through by 0063 (926) 746 2081(globe telecom) for your opinions and questions or visit our offical blog to want to know last annoucement and updates on site.

Last Modify Date : 7/4/2009 6:33:18 PM (...don't count your chickens before they hatch)