Log In Sign Up

mpmoran.co.uk
robots.txt

Robots Exclusion Standard data for mpmoran.co.uk

Resource Scan

Scan Details

Site Domain mpmoran.co.uk
Base Domain mpmoran.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-03-26T13:45:46+00:00
Next Scan 2026-04-25T13:45:46+00:00

Last Successful Scan

Scanned2026-02-02T08:09:52+00:00
URL https://mpmoran.co.uk/robots.txt
Redirect https://www.mpmoran.co.uk/robots.txt
Redirect Domain www.mpmoran.co.uk
Redirect Base mpmoran.co.uk
Domain IPs 104.26.12.62, 104.26.13.62, 172.67.69.124, 2606:4700:20::681a:c3e, 2606:4700:20::681a:d3e, 2606:4700:20::ac43:457c
Redirect IPs 104.26.12.62, 104.26.13.62, 172.67.69.124, 2606:4700:20::681a:c3e, 2606:4700:20::681a:d3e, 2606:4700:20::ac43:457c
Response IP 104.26.13.62
Found Yes
Hash b1021100dc39bbe1e0fe22196783eccee0cb123e89cf38bf7e91e29d2c78de6c
SimHash a12451460718

Groups

facebookcatalog/1.0

Rule Path
Disallow

*

Rule Path
Allow /pub/static/*.css$
Allow /pub/static/*.js$
Allow /static/*.css$
Allow /static/*.js$
Allow /media/catalog/product/
Allow /media/wysiwyg/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /sendfriend/
Disallow /review/
Disallow /wishlist/
Disallow /newsletter/
Disallow /contact/
Disallow /tag/
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /search/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pub/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /errors/
Disallow /index.php/
Disallow /static/
Disallow /pub/errors/
Disallow /pub/opt/
Disallow /pub/report/
Disallow /pub/var/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /cdn-cgi/
Disallow /*.git
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$
Disallow /search-results
Disallow /blog/tag/
Back to top

Other Records

Field Value
sitemap https://www.mpmoran.co.uk/media/sitemap/sitemap2.xml
sitemap https://www.mpmoran.co.uk/media/sitemap/blogsitemap.xml
sitemap https://www.mpmoran.co.uk/media/sitemap/sitemap2.xml
Back to top

Comments

  • Exception: allow everything for Facebook Catalog crawler
  • Allow important pages
  • Disallow sensitive
  • Disable checkout & customer account
  • Disable search pages
  • Disallow: /catalogsearch/
  • Disable common folders
  • Disable common files
  • Disallow URL params that cause duplicate content
  • Disallow: /*?dir=
  • Disallow: /*?limit=
  • Disallow: /*?mode=
  • Disallow: /*?price=
  • Disallow: /*?SID=
  • Disallow: /*?___from_store=
  • Disallow: /*?___store=
  • Disallow: /*?___SID=
  • Disable sorting (Avoid duplicate content)
  • Disallow: /*?*product_list_order=
  • Disallow: /*?*product_list_limit=
  • Disable version control folders and others
Back to top