Log In Sign Up

misupplies.co.uk
robots.txt

Robots Exclusion Standard data for misupplies.co.uk

Resource Scan

Scan Details

Site Domain misupplies.co.uk
Base Domain misupplies.co.uk
Scan Status Ok
Last Scan2025-08-22T07:55:18+00:00
Next Scan 2025-09-21T07:55:18+00:00

Last Scan

Scanned2025-08-22T07:55:18+00:00
URL https://misupplies.co.uk/robots.txt
Redirect https://www.misupplies.co.uk/robots.txt
Redirect Domain www.misupplies.co.uk
Redirect Base misupplies.co.uk
Domain IPs 109.108.148.102
Redirect IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.80.1
Found Yes
Hash 6f7ee3a380c6c27deb055562f11b63e31df57650cba07b1b36f015f89c9d3425
SimHash 700fd670c61a

Groups

etaospider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

brandverity

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

pricebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

charlotte

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

seznambot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

Rule Path
Allow /
Disallow /ajax/removefrombasket/
Disallow /basket$
Disallow /checkout/
Disallow /errors/
Disallow /quickbuy/
Disallow /product_personalisation/
Disallow /register/
Disallow /register
Disallow /vs/
Disallow /images/products/yofla/
Disallow /customer_groups/choose_group/
Disallow /formbuilder/
Disallow /callbacks/
Disallow /ajax/formatprice/
Disallow /products/
Disallow /search/

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap https://www.misupplies.co.uk/sitemap-index.xml
Back to top

Warnings

  • `request-rate` is not a known field.
Back to top