potguide.com
robots.txt

Robots Exclusion Standard data for potguide.com

Resource Scan

Scan Details

Site Domain potguide.com
Base Domain potguide.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-09T03:44:35+00:00
Next Scan 2024-12-08T03:44:35+00:00

Last Successful Scan

Scanned2024-07-18T22:53:34+00:00
URL https://potguide.com/robots.txt
Domain IPs 3.97.87.22, 3.99.24.167
Response IP 3.99.24.167
Found Yes
Hash 9afc2e7abe704885ee4f024825500f219a0df1844d01bed9114821b2be0833af
SimHash 21025b60ce56

Groups

*

Rule Path
Disallow /aspnet_client/
Disallow /bin/
Disallow /config/
Disallow /data/
Disallow /macroScripts/
Disallow /umbraco_client/
Disallow /usercontrols/
Disallow /xslt/

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

mauibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://potguide.com/sitemap/

Comments

  • robots.txt for Umbraco
  • moz spider throttle
  • SemrushBot spider throttle
  • MauiBot block