robots.txt in wordpress

WordPress robots.txt file implementations with directives and rules to guide search engine bots to the pages you want them to crawl.