# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 # The default is to block all access as we don't want crawlers crawling development server User-agent: * Disallow: /structure/ Disallow: /personal/ Disallow: /clients/