User-agent: * Noindex: /user/ Disallow: /user/ Allow: / Crawl-delay: 45 User-agent: Slurp Disallow: /user/ Allow: / Crawl-delay: 0.6