# All robots will spider the domain User-agent: * Disallow: # Disallow directory /css User-agent: * # Disallow directory /websites User-agent: * # Disallow directory /newsletter User-agent: *