.Google.com's John Mueller answered a question about why Google.com indexes pages that are prohibited from creeping by robots.txt and why the it's risk-free to overlook the relevant Search Console documents concerning those creeps.Robot Web Traffic To Question Parameter URLs.The person talking to the inquiry documented that bots were actually developing web links to non-existent query parameter URLs (? q= xyz) to pages with noindex meta tags that are actually likewise blocked in robots.txt. What caused the concern is actually that Google is actually crawling the links to those pages, receiving shut out through robots.txt (without watching a noindex robotics meta tag) at that point receiving turned up in Google Look Console as "Indexed, though blocked out by robots.txt.".The person talked to the observing concern:." Yet below is actually the major concern: why would certainly Google mark webpages when they can't also find the content? What is actually the advantage during that?".Google.com's John Mueller confirmed that if they can't crawl the web page they can not observe the noindex meta tag. He additionally helps make an interesting reference of the site: hunt operator, suggesting to dismiss the results because the "typical" consumers won't find those end results.He wrote:." Yes, you're right: if we can not crawl the webpage, we can't view the noindex. That pointed out, if we can not crawl the pages, at that point there is actually certainly not a great deal for our company to mark. Therefore while you could observe a few of those web pages with a targeted website:- concern, the ordinary consumer won't view them, so I definitely would not bother it. Noindex is likewise great (without robots.txt disallow), it merely suggests the Links will wind up being actually crept (as well as wind up in the Look Console document for crawled/not recorded-- neither of these statuses induce concerns to the rest of the website). The important part is actually that you don't make them crawlable + indexable.".Takeaways:.1. Mueller's answer affirms the restrictions in using the Website: hunt progressed search operator for diagnostic main reasons. Some of those main reasons is actually given that it's certainly not attached to the routine hunt mark, it's a distinct trait entirely.Google's John Mueller commented on the website search driver in 2021:." The brief solution is actually that a web site: inquiry is certainly not implied to be comprehensive, neither made use of for diagnostics functions.A website inquiry is actually a certain kind of search that confines the end results to a particular internet site. It's basically merely words website, a colon, and afterwards the internet site's domain name.This concern limits the outcomes to a certain internet site. It's certainly not implied to be an extensive compilation of all the pages from that internet site.".2. Noindex tag without making use of a robots.txt is fine for these type of circumstances where a crawler is connecting to non-existent web pages that are receiving discovered through Googlebot.3. Links with the noindex tag are going to create a "crawled/not listed" item in Search Console and also those will not have an adverse result on the remainder of the internet site.Read through the question as well as answer on LinkedIn:.Why would certainly Google.com index web pages when they can't also observe the material?Included Picture through Shutterstock/Krakenimages. com.