# # All robots will spider the domain User-agent: psbot Disallow: / User-agent: Googlebot-Image Disallow: / User-agent: ia_archiver Disallow: / User-agent: asterias Disallow: / User-agent: yahoo-blogs/v3.9 Disallow: / User-agent: yahoo-mmcrawler Disallow: / User-agent: * Disallow: /album/ Disallow: /security/ Disallow: /admin/ Disallow: