EN The robots.txt controls the access of the crawler to some areas of the website. This sometimes can be risky, especially if the GoogleBot is accidentally not allowed to crawl the entire site, but there are situations where a robots.txt file can be handy.