albertgoodman.co.uk
robots.txt

Robots Exclusion Standard data for albertgoodman.co.uk

Resource Scan

Scan Details

Site Domain albertgoodman.co.uk
Base Domain albertgoodman.co.uk
Scan Status Ok
Last Scan2024-05-11T04:59:51+00:00
Next Scan 2024-06-10T04:59:51+00:00

Last Scan

Scanned2024-05-11T04:59:51+00:00
URL https://albertgoodman.co.uk/robots.txt
Domain IPs 192.124.249.84
Response IP 192.124.249.84
Found Yes
Hash 492b145993754f292f350bd32bf8a04562f83bddc63bc0b4efbbbf38aa3acbe8
SimHash 63381d566691

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://albertgoodman.co.uk/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://albertgoodman.co.uk/
  • live - don't allow web crawlers to index cpresources/ or vendor/