# robots.txt - http://www.pays-aisne.org User-agent: msnbot Crawl-delay: 15 User-agent: slurp Crawl-delay: 10 User-agent: * Disallow: /usages/ Disallow: /webalizer/