oldies.org.uk
robots.txt

Robots Exclusion Standard data for oldies.org.uk

Resource Scan

Scan Details

Site Domain oldies.org.uk
Base Domain oldies.org.uk
Scan Status Ok
Last Scan2025-09-09T11:48:48+00:00
Next Scan 2025-10-09T11:48:48+00:00

Last Scan

Scanned2025-09-09T11:48:48+00:00
URL https://oldies.org.uk/robots.txt
Domain IPs 192.124.249.29
Response IP 192.124.249.29
Found Yes
Hash 1afe366af0cc5d45f0e511e14ea89d31de2eb1fc70372cb3fb387dd73ffffbfb
SimHash 6410d9337315

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-content/

googlebot-image

Rule Path
Disallow /wp-content/*

googlebot

Rule Path
Disallow /wp-content/*
Allow /wp-content/themes/oldies2015

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100