thelaptoppowersupplyshop.co.uk
robots.txt

Robots Exclusion Standard data for thelaptoppowersupplyshop.co.uk

Resource Scan

Scan Details

Site Domain thelaptoppowersupplyshop.co.uk
Base Domain thelaptoppowersupplyshop.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-14T06:34:44+00:00
Next Scan 2024-10-14T06:34:44+00:00

Last Successful Scan

Scanned2024-08-15T21:33:47+00:00
URL https://thelaptoppowersupplyshop.co.uk/robots.txt
Redirect https://www.thelaptoppowersupplyshop.co.uk/robots.txt
Redirect Domain www.thelaptoppowersupplyshop.co.uk
Redirect Base thelaptoppowersupplyshop.co.uk
Domain IPs 213.171.211.66
Redirect IPs 213.171.211.66
Response IP 213.171.211.66
Found Yes
Hash 1b9a211458f216d3599927011f9f9b19fac9683d9749b0987a894e1be407a760
SimHash 8beb7c5387e0

Groups

linguee

Rule Path
Disallow /

lcc

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

exabot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

buck

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

*

Rule Path
Disallow /*?sort
Disallow /*%26sort
Disallow /*?limit
Disallow /*%26limit
Disallow /*?route=checkout
Disallow /*?route=account
Disallow /*?route=product%2Fsearch
Disallow /*?route=product%2Fproduct%2Fattribute_info
Disallow /*?route=product%2Fmanufacturer%2Fsummary
Disallow /*?page=1
Disallow /*CUSTOMERREVIEW
Disallow /admin/
Allow /

Comments

  • Crawl-delay: 5