Log In Sign Up

moorelarge.co.uk
robots.txt

Robots Exclusion Standard data for moorelarge.co.uk

Resource Scan

Scan Details

Site Domain moorelarge.co.uk
Base Domain moorelarge.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-10-12T11:06:38+00:00
Next Scan 2026-01-10T11:06:38+00:00

Last Successful Scan

Scanned2024-08-26T08:24:43+00:00
URL https://moorelarge.co.uk/robots.txt
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 151.101.193.124
Found Yes
Hash bb1a36bd0de0ec04f2da28e7f92b5813a8bb90c77a439b32420116a8d5f5a4df
SimHash a714f9560662

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dynamictemplates

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow /index.php/
Disallow /wishlist/
Disallow /admin/
Disallow /review/product/
Disallow /sendfriend/
Disallow /sendfriend/product/
Disallow /enable-cookies/
Disallow /directory/
Disallow /api.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /get.php
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /README.txt
Disallow /RELEASE_NOTES.txt
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /customer/account/create/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /*?
Disallow /*?dir*
Disallow /*?dir=desc
Disallow /*?dir=asc
Disallow /stores/store/redirect/
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /cgi-bin/
Disallow /cleanup.php
Disallow /apc.php
Disallow /memcache.php
Disallow /phpinfo.php
Disallow /*?SID=
Back to top

Other Records

Field Value
sitemap https://www.moorelarge.co.uk/media/sitemap.xml
Back to top

Comments

  • Moore Large Robots.txt file
  • Enable robots.txt rules for all crawlers
  • Do not crawl common Magento folders
  • Do not crawl common Magento files
  • Do not crawl checkout and user account pages
  • Do not crawl search pages and non SEO optimized catalog links
  • Blocking filter options, parameters and pagination
  • Blocking CMS directories
  • Do not crawl common server technical folders and files
  • Do not crawl links with session IDs
Back to top