productplan.com
robots.txt

Robots Exclusion Standard data for productplan.com

Resource Scan

Scan Details

Site Domain productplan.com
Base Domain productplan.com
Scan Status Ok
Last Scan2024-10-28T05:31:39+00:00
Next Scan 2024-11-27T05:31:39+00:00

Last Scan

Scanned2024-10-28T05:31:39+00:00
URL https://productplan.com/robots.txt
Redirect https://www.productplan.com/robots.txt
Redirect Domain www.productplan.com
Redirect Base productplan.com
Domain IPs 141.193.213.10, 141.193.213.11
Redirect IPs 141.193.213.10, 141.193.213.11
Response IP 141.193.213.11
Found Yes
Hash 6db3539cc8822b617cb71ccb7f88d92b28e0c8e3e8ec0f4e0e11d96f355d36f3
SimHash 2638e049c770

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /agreement/

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Comments

  • Don't allow bots to hit our admin pages and our E Plus TOS
  • Slow MOZ down so that we don't get 504s