#Set access requests for spiders and search engines User-agent: * # applies to all robots Disallow: # disallowing indexing of all pages