Log In Sign Up

walkingclub.org.uk
robots.txt

Robots Exclusion Standard data for walkingclub.org.uk

Resource Scan

Scan Details

Site Domain walkingclub.org.uk
Base Domain walkingclub.org.uk
Scan Status Ok
Last Scan2024-11-15T16:52:28+00:00
Next Scan 2024-11-22T16:52:28+00:00

Last Scan

Scanned2024-11-15T16:52:28+00:00
URL https://walkingclub.org.uk/robots.txt
Redirect https://www.walkingclub.org.uk/robots.txt
Redirect Domain www.walkingclub.org.uk
Redirect Base walkingclub.org.uk
Domain IPs 35.189.67.104
Redirect IPs 35.189.67.104
Response IP 35.189.67.104
Found Yes
Hash 5437e3c1ea8ee48ffea22512c600dfcf69aa2750276fba73bbb7aeb977d6b889
SimHash 5b1c3d400cf8

Groups

ahrefsbot
mj12bot
semrushbot
linkdexbot
wesee
smtbot
mail.ru_bot
exabot
grapeshotcrawler
maxpointcrawler
spbot
megaindex.ru
blexbot
heritrix
criteobot/0.1
ias-sg
awariobot

Rule Path
Disallow /

*

Rule Path
Disallow /test
Disallow /depository
Disallow /ssi
Disallow /cgi-bin
Disallow /swc/admin
Back to top

Other Records

Field Value
sitemap https://www.walkingclub.org.uk/sitemap.xml
Back to top

Comments

  • Notice
  • If you would like to crawl us, you contact can us here: http://www.walkingclub.org.uk/site/contact.shtml
  • cut xxxxx -d'"' -f6 | sort | uniq -c|sort -n
  • NO ACCESS
  • RESTRICTED ACCESS
Back to top

Warnings

  • `host` is not a known field.
Back to top