Log In Sign Up

saashub.com
robots.txt

Robots Exclusion Standard data for saashub.com

Resource Scan

Scan Details

Site Domain saashub.com
Base Domain saashub.com
Scan Status Ok
Last Scan2024-11-09T16:46:54+00:00
Next Scan 2024-11-16T16:46:54+00:00

Last Scan

Scanned2024-11-09T16:46:54+00:00
URL https://saashub.com/robots.txt
Redirect https://www.saashub.com/robots.txt
Redirect Domain www.saashub.com
Redirect Base saashub.com
Domain IPs 172.66.40.205, 172.66.43.51, 2606:4700:3108::ac42:28cd, 2606:4700:3108::ac42:2b33
Redirect IPs 172.66.40.205, 172.66.43.51, 2606:4700:3108::ac42:28cd, 2606:4700:3108::ac42:2b33
Response IP 172.66.40.205
Found Yes
Hash 8853ba4eab710d820f2ad032cc4552cead85576bcfdecb4b47c41b366d0f4b91
SimHash 13b49d053f42

Groups

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.saashub.com/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
Back to top