# robots.txt for http://example.com/ User-agent: * Crawl-delay: 120 # uncomment this after replacing HTTP_HOST # Sitemap: http://example.com/sitemap.xml