Log In Sign Up

airthings.com
robots.txt

Robots Exclusion Standard data for airthings.com

Resource Scan

Scan Details

Site Domain airthings.com
Base Domain airthings.com
Scan Status Ok
Last Scan2024-05-08T05:12:43+00:00
Next Scan 2024-05-22T05:12:43+00:00

Last Scan

Scanned2024-05-08T05:12:43+00:00
URL https://airthings.com/robots.txt
Redirect https://www.airthings.com/robots.txt
Redirect Domain www.airthings.com
Redirect Base airthings.com
Domain IPs 108.156.120.102, 108.156.120.120, 108.156.120.129, 108.156.120.36
Redirect IPs 2600:9000:2200:1000:12:2906:b540:93a1, 2600:9000:2200:1e00:12:2906:b540:93a1, 2600:9000:2200:4600:12:2906:b540:93a1, 2600:9000:2200:6800:12:2906:b540:93a1, 2600:9000:2200:8200:12:2906:b540:93a1, 2600:9000:2200:8600:12:2906:b540:93a1, 2600:9000:2200:bc00:12:2906:b540:93a1, 2600:9000:2200:f000:12:2906:b540:93a1, 54.192.18.20, 54.192.18.35, 54.192.18.6, 54.192.18.82
Response IP 13.33.88.91
Found Yes
Hash 3030e05fbf676a9343ab8223551767833bb55248b54b19c9337e4ef2e12835d0
SimHash 6c66b968a190

Groups

*

Rule Path
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pkginfo/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow */review/
Disallow */catalog/category/view/
Disallow */catalog/product/view/
Disallow */sample-*
Disallow */blog/sample-*
Disallow */_hcms/preview/
Disallow */hs/manage-preferences/
Disallow */checkout/*
Disallow */customer/*
Disallow */wishlist/*
Disallow /*SID%3D
Back to top

Other Records

Field Value
sitemap https://www.airthings.com/media/sitemap_b2c_CA_EN_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_CA_EN_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_CA_FR_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_CA_FR_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_DA_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_DA_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_FR_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_FR_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_DE_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_DE_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_IT_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_IT_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_ES_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_ES_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_NO_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_NO_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_SV_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_SV_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_EN_GB_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_EN_GB_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_US_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_US_product.xml
sitemap https://www.airthings.com/media/sitemap_b2c_global_category.xml
sitemap https://www.airthings.com/media/sitemap_b2c_global_product.xml
sitemap https://www.airthings.com/media/sitemap_additional_link.xml
Back to top

Comments

  • Crawlers Setup
  • Directories
  • Paths (clean URLs)
  • Paths (not clean URLs)
  • Website Sitemaps
Back to top