You want that some of your Web pages should not be crawled. Which one of the following options will you use to accomplish the task?

There is no standard HTML tag called 'NO Crawl' - while a meta robots tag with content='noindex' exists, it only signals the crawler not to index the page after it has already been fetched, and does not prevent crawling.

There is no standard file named 'private.txt' recognized by web crawlers - only robots.txt is specified in the Robots Exclusion Protocol.

The robots.txt file, placed in the root directory of a web server, uses the Robots Exclusion Protocol to communicate crawling restrictions to compliant web crawlers via Disallow directives for specific paths. This prevents those pages from being crawled or indexed and is the universally accepted standard supported by all major search engine crawlers. Any web page listed under a Disallow directive will be skipped by robots that honor the protocol.

Enabling SSL/TLS secures data in transit between the browser and server but does not instruct or restrict crawlers from accessing web pages.