prisons.org.uk
robots.txt

Robots Exclusion Standard data for prisons.org.uk

Resource Scan

Scan Details

Site Domain prisons.org.uk
Base Domain prisons.org.uk
Scan Status Ok
Last Scan2025-10-29T11:17:29+00:00
Next Scan 2025-11-28T11:17:29+00:00

Last Scan

Scanned2025-10-29T11:17:29+00:00
URL https://prisons.org.uk/robots.txt
Domain IPs 104.16.150.108, 104.16.151.108, 2606:4700::6810:966c, 2606:4700::6810:976c
Response IP 104.16.151.108
Found Yes
Hash 647e9afd376f1014f5c48c96e3eb8ff3b8530db0f2cb3a112c51334302d9e589
SimHash ca1fc0c2c890

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

Rule Path
Disallow /wp-admin/

*

Rule Path
Disallow /wp-content/

honolulu-bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

*

Rule Path
Disallow /newspaper/