m-w.co.uk
robots.txt

Robots Exclusion Standard data for m-w.co.uk

Resource Scan

Scan Details

Site Domain m-w.co.uk
Base Domain m-w.co.uk
Scan Status Ok
Last Scan2024-11-13T13:22:08+00:00
Next Scan 2024-11-20T13:22:08+00:00

Last Scan

Scanned2024-11-13T13:22:08+00:00
URL https://m-w.co.uk/robots.txt
Redirect https://www.m-w.co.uk/robots.txt
Redirect Domain www.m-w.co.uk
Redirect Base m-w.co.uk
Domain IPs 143.42.254.148
Redirect IPs 23.209.46.12, 23.209.46.25
Response IP 23.45.207.166
Found Yes
Hash 49c5e74555304b08de53079b93b5337faee164dc1a0ad76ea0280e5753d9f975
SimHash 0348197627b2

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://www.m-w.co.uk/sitemaps-1-sitemap.xml
sitemap https://www.porteighty.co.uk/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.m-w.co.uk/
  • live - don't allow web crawlers to index cpresources/ or vendor/