Log In Sign Up

staffordshire.gov.uk
robots.txt

Robots Exclusion Standard data for staffordshire.gov.uk

Resource Scan

Scan Details

Site Domain staffordshire.gov.uk
Base Domain staffordshire.gov.uk
Scan Status Ok
Last Scan2024-06-03T21:46:25+00:00
Next Scan 2024-06-10T21:46:25+00:00

Last Scan

Scanned2024-06-03T21:46:25+00:00
URL https://staffordshire.gov.uk/robots.txt
Redirect https://www.staffordshire.gov.uk/robots.txt
Redirect Domain www.staffordshire.gov.uk
Redirect Base staffordshire.gov.uk
Domain IPs 185.18.139.141
Redirect IPs 185.18.139.141
Response IP 185.18.139.141
Found Yes
Hash 9ca8b1ecfd6464052e54a79602b04869951a66c0a7bca98b5f27ac6daa7acb03
SimHash 68bc05933ee7

Groups

*

Product Comment
* match all robots

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://www.staffordshire.gov.uk/sitemap.xml
Back to top

Comments

  • RAH - 09/08/2022 Contents of old file below, seems to refer to it being used only on preview
  • Do NOT publish to Live, only for Preview site.
  • Disallow: / # don't allow ANY spidering
Back to top

Warnings

  • 19 invalid lines.
Back to top