User-agent: * Allow: / # disallow stress test User-agent: stress-agent Disallow: User-agent: wget Disallow: / # exclude help system from robots User-agent: * Disallow: /cgi-bin/ Disallow: /forum/ Disallow: /DE/img/ Disallow: /DE/tiras/ Disallow: /DE/ES/img/ Disallow: /mafalda/pict/