# This is a robots.txt file that can be used to indicate to # WWW robots which parts of a web site should not be indexed. # # All robots should stay away from these folders User-agent: * Disallow: /cgi-bin/ Disallow: /js/ Disallow: /css/ Disallow: /images/