# Robots.txt for www.salvationarmy.org # Revised 19/1/2007 # Stop Yahoo! Slurp indexing outdated static and text-only links User-agent: Slurp Disallow: /*/stc-* Disallow: /*/txt-* # Stop Yahoo! Slurp indexing outdated static and text-only links User-agent: Yahoo! Slurp Disallow: /*/stc-* Disallow: /*/txt-* # Stop ConveraCrawler indexing anything User-agent: ConveraCrawler Disallow: / # Deal with MSN bot User-agent: msnbot Disallow: /*/stc-* Disallow: /*/txt-* # Deal with Googlebot User-agent: googlebot Disallow: /*/stc-* Disallow: /*/txt-* # Deal with Scirus User-agent: Scirus Disallow: / User-agent: Fast Disallow: / User-agent: FAST-WebCrawler Disallow: / # Deal with some other annoyances User-agent: szukacz Disallow: /