offeroftheday.co.uk
robots.txt

Robots Exclusion Standard data for offeroftheday.co.uk

Resource Scan

Scan Details

Site Domain offeroftheday.co.uk
Base Domain offeroftheday.co.uk
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-24T02:51:23+00:00
Next Scan 2024-10-01T02:51:23+00:00

Last Successful Scan

Scanned2024-09-16T02:50:32+00:00
URL https://www.offeroftheday.co.uk/robots.txt
Domain IPs 188.65.36.177
Response IP 188.65.36.177
Found Yes
Hash 1a49a32e9c0c1a9e89f49cb82cb7897c7eb1c4a833c104586c66554f4ac30911
SimHash 624e8af8fa25

Groups

mediapartners-google

Rule Path
Allow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

magpie-crawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow /*?order=*
Disallow /todays.xml
Disallow /redirect.asp
Disallow /error400.html
Disallow /error401.html
Disallow /error403.html
Disallow /error404.html
Disallow /error404.asp
Disallow /error410.asp
Disallow /error500.html
Disallow /model.asp
Disallow /model_ean.asp
Disallow /voucher_info.asp
Disallow /adv.asp
Disallow /report.asp
Disallow /promo.asp
Disallow /friend.asp
Disallow /no-store.asp
Disallow /category2.asp
Disallow /xmas.asp
Disallow /voucher-products.asp
Disallow /store-vouchers.asp
Disallow /sections/more.asp
Disallow /report.asp
Disallow /sections/pages.asp
Disallow /sections/show_cats.asp
Disallow /sections/fast_register.asp
Disallow /product.asp
Disallow /functions/
Disallow /help/

Other Records

Field Value
sitemap https://www.offeroftheday.co.uk/sitemap.xml