Log In Sign Up

wokingsc.co.uk
robots.txt

Robots Exclusion Standard data for wokingsc.co.uk

Resource Scan

Scan Details

Site Domain wokingsc.co.uk
Base Domain wokingsc.co.uk
Scan Status Ok
Last Scan2025-08-29T06:30:25+00:00
Next Scan 2025-09-12T06:30:25+00:00

Last Scan

Scanned2025-08-29T06:30:25+00:00
URL https://wokingsc.co.uk/robots.txt
Redirect https://www.wokingsc.co.uk/robots.txt
Redirect Domain www.wokingsc.co.uk
Redirect Base wokingsc.co.uk
Domain IPs 104.18.159.233, 104.18.160.233
Redirect IPs 104.18.159.233, 104.18.160.233
Response IP 104.18.160.233
Found Yes
Hash 35a43f88b9930c6559503cc7f75d45c791a676391f169b2443a63920d7d71fe7
SimHash 8dd3d421f5d1

Groups

*

Rule Path
Disallow /

mediapartners-google
google-display-ads-bot
adsbot-google-mobile
googlebot
bingbot
slurp
duckduckbot
yandexbot
facebot
applebot

Rule Path
Disallow /*-del
Back to top

Comments

  • This robots.txt file disallows all engines/crawlers/indexers by default.
  • Global directives, but not always obeyed.
  • Named directive(s) specifically disallowing the most common engines/crawlers/indexers.
  • Prevent the indexing of any sites that are no longer active but still accessible online.
Back to top