Log In Sign Up

iqair.com
robots.txt

Robots Exclusion Standard data for iqair.com

Resource Scan

Scan Details

Site Domain iqair.com
Base Domain iqair.com
Scan Status Ok
Last Scan2024-11-09T08:37:03+00:00
Next Scan 2024-11-16T08:37:03+00:00

Last Scan

Scanned2024-11-09T08:37:03+00:00
URL https://iqair.com/robots.txt
Domain IPs 13.33.88.124, 13.33.88.13, 13.33.88.55, 13.33.88.83
Response IP 13.33.88.13
Found Yes
Hash 5b726e675e68ac1df3c7ff4ad88cdf48210080b3561d1ebe1bc089a287f5cc2b
SimHash a8149d01c765

Groups

facebookexternalhit

Rule Path
Disallow /*.json$

Other Records

Field Value
crawl-delay 5

claudebot

Rule Path
Allow /

gptbot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

applebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

googleother

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

screaming frog seo spider

Rule Path
Allow /

twitterbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandexbot

Rule Path
Allow /

*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.iqair.com/sitemap.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
Back to top