Log In Sign Up

canadianponcho.activeboard.com
robots.txt

Robots Exclusion Standard data for canadianponcho.activeboard.com

Resource Scan

Scan Details

Site Domain canadianponcho.activeboard.com
Base Domain activeboard.com
Scan Status Ok
Last Scan2024-09-18T05:18:17+00:00
Next Scan 2024-09-25T05:18:17+00:00

Last Scan

Scanned2024-09-18T05:18:17+00:00
URL https://canadianponcho.activeboard.com/robots.txt
Domain IPs 64.34.33.212
Response IP 64.34.33.212
Found Yes
Hash f1ca33c92545e6dedb294435385a20d614d3a7ac7c680c2f58c6e986300d41aa
SimHash 23acff5561b2

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /ab_monitor
Back to top

Comments

  • Yahoo! Slurp
  • SemrushBot (General scraping)
  • SemrushBot-SA (SEO / technical crawling)
  • MJ12bot (https://mj12bot.com/)
  • AhrefsBot (https://ahrefs.com/robot)
  • All
Back to top