# # This is the robots.txt file. It should be in the root (/) # of your web site. # # This file is used to tell search engine crawlers what # directories they should ignore when indexing. # # Lines beginning with # are comments, and are not processed # by crawlers. # # The format is a series of records. Each record begins with # a User-agent: statement, and is followed by at least one # Disallow: statement. The preset records below demonstrate # this format. # # Records are separated by blank lines # # For further information, visit # http://info.webcrawler.com/mak/projects/robots/norobots.html # # Edit this record as appropriate for your site. Add any # other directories you want crawlers to ignore. # robots.txt generated at http://www.mcanerin.com User-agent: googlebot-image Disallow: / User-agent: psbot Disallow: / User-agent: asterias Disallow: / User-agent: yahoo-blogs/v3.9 Disallow: / User-agent: * Disallow: Crawl-delay: 5 Disallow: /cgi-bin/ Disallow: /admin/ Disallow: /dev/ Sitemap: http://www.abledata.com/abledata.cfm?pageid=19323