lutontown.co.uk
robots.txt

Robots Exclusion Standard data for lutontown.co.uk

Resource Scan

Scan Details

Site Domain lutontown.co.uk
Base Domain lutontown.co.uk
Scan Status Ok
Last Scan2024-05-22T21:15:10+00:00
Next Scan 2024-05-29T21:15:10+00:00

Last Scan

Scanned2024-05-22T21:15:10+00:00
URL https://lutontown.co.uk/robots.txt
Redirect https://www.lutontown.co.uk/robots.txt
Redirect Domain www.lutontown.co.uk
Redirect Base lutontown.co.uk
Domain IPs 217.114.94.2
Redirect IPs 104.18.33.7, 172.64.154.249, 2606:4700:4400::6812:2107, 2606:4700:4400::ac40:9af9
Response IP 104.18.33.7
Found Yes
Hash 90aafbca4d678532feefbcb9800a7a5d2afd3d3345eb9e9a6beeccc089eb6272
SimHash 6316f055c514

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.lutontown.co.uk/sitemap.xml