# Disallow requests from TurnitinBot User-agent: TurnitinBot Disallow: / # Disallow requests from Zealbot User-agent: Zealbot Disallow: / # Disallow some requests from Slurp (Inktomi) so they dont see cross-links User-agent: Slurp Crawl-delay: 15 Disallow: /about/ Disallow: /examples/ User-agent: msnbot Crawl-delay: 5 Disallow: /iman/ Disallow: /requestinfo/ Disallow: /r/ Disallow: /about/friend.php Disallow: /about/friendsend.php Disallow: /about/tou.php Disallow: /offsite.php # Allow requests from other robots, except to the disallowed dirs and files User-agent: * Crawl-delay: 5 Disallow: /iman/ Disallow: /requestinfo/ Disallow: /r/ Disallow: /about/friend.php Disallow: /about/friendsend.php Disallow: /about/tou.php Disallow: /offsite.php Disallow: /jump/ Disallow: /lp-next/ Disallow: /ccsearch Disallow: /ccsearch/ Disallow: /*/em-* Disallow: /faqs/online-mba-L2.php Sitemap: http://www.allpsychologyschools.com/google-sitemap.php