harrisonandhetherington.co.uk
robots.txt

Robots Exclusion Standard data for harrisonandhetherington.co.uk

Resource Scan

Scan Details

Site Domain harrisonandhetherington.co.uk
Base Domain harrisonandhetherington.co.uk
Scan Status Ok
Last Scan2025-03-27T02:19:57+00:00
Next Scan 2025-04-26T02:19:57+00:00

Last Scan

Scanned2025-03-27T02:19:57+00:00
URL https://harrisonandhetherington.co.uk/robots.txt
Domain IPs 104.21.66.206, 172.67.207.65, 2606:4700:3032::ac43:cf41, 2606:4700:3033::6815:42ce
Response IP 104.21.66.206
Found Yes
Hash e47f2af6349182d81cf3930bd5f782075db64069deb101ba96382028c6920a54
SimHash c09e5e6068ba

Groups

*

Rule Path
Disallow /extensions/
Disallow /manifest/
Disallow /symphony/
Disallow /vendor/
Disallow /maintenance/
Disallow /livestock/farm-to-farm/submit-2/success/
Disallow /livestock/farm-to-farm/submit-2/success/*
Disallow /entry-forms/pedigree-beef-online/success/
Disallow /entry-forms/pedigree-beef-online/success/*
Disallow /entry-forms/pedigree-dairy-online/success
Disallow /entry-forms/pedigree-dairy-online/success/*
Disallow /machinery/items-for-sale-privately/submit/success/
Disallow /machinery/items-for-sale-privately/submit/success/*
Disallow /machinery/entry-forms-2/success/
Disallow /machinery/entry-forms-2/success/*
Disallow /admin-top-images/
Disallow /admin-focus-boxes/
Disallow /admin-icon-boxes/
Disallow /admin-banners/
Disallow /admin-beef-bull-entries/
Disallow /admin-beef-female-entries/
Disallow /admin-dairy-entries/
Disallow /admin-machinery-entries/
Disallow /search/
Disallow /sitemapxml/

Other Records

Field Value
crawl-delay 5

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /