Log In Sign Up

activeadventures.com
robots.txt

Robots Exclusion Standard data for activeadventures.com

Resource Scan

Scan Details

Site Domain activeadventures.com
Base Domain activeadventures.com
Scan Status Ok
Last Scan2025-06-12T04:30:47+00:00
Next Scan 2025-07-12T04:30:47+00:00

Last Scan

Scanned2025-06-12T04:30:47+00:00
URL https://activeadventures.com/robots.txt
Domain IPs 172.66.41.32, 172.66.42.224, 2606:4700:3108::ac42:2920, 2606:4700:3108::ac42:2ae0
Response IP 172.66.41.32
Found Yes
Hash 494742e371ae40305d2482854e4b7f9d441543943bafc8753130c1135e6a4477
SimHash 6c60de62aea2

Groups

*

Rule Path
Disallow /admin
Disallow /Security/

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /admin
Disallow /Security/

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

swiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

claude-web

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://activeadventures.com/sitemap.xml
sitemap https://austinadventures.com/sitemap.xml
Back to top

Comments

  • Allow/disallow rules for https://activeadventures.com
  • Allow/disallow rules and tracking for https://austinadventures.com
  • Specific bot crawl delays for activeadventures.com
Back to top

Warnings

  • `host` is not a known field.
Back to top