Log In Sign Up

chewvalleytrees.co.uk
robots.txt

Robots Exclusion Standard data for chewvalleytrees.co.uk

Resource Scan

Scan Details

Site Domain chewvalleytrees.co.uk
Base Domain chewvalleytrees.co.uk
Scan Status Ok
Last Scan2025-05-20T02:33:37+00:00
Next Scan 2025-06-19T02:33:37+00:00

Last Scan

Scanned2025-05-20T02:33:37+00:00
URL https://chewvalleytrees.co.uk/robots.txt
Redirect https://www.chewvalleytrees.co.uk/robots.txt
Redirect Domain www.chewvalleytrees.co.uk
Redirect Base chewvalleytrees.co.uk
Domain IPs 104.21.8.220, 172.67.140.73, 2606:4700:3032::ac43:8c49, 2606:4700:3035::6815:8dc
Redirect IPs 104.21.8.220, 172.67.140.73, 2606:4700:3032::ac43:8c49, 2606:4700:3035::6815:8dc
Response IP 172.67.140.73
Found Yes
Hash 97225b74814abb42d04de1407d97a8ed70c5f179d65a01dbf03645b05cee7e7a
SimHash 2b2a19726e9a

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow /*?*growth-rate=
Back to top

Other Records

Field Value
sitemap https://www.chewvalleytrees.co.uk/sitemaps-1-sitemap.xml
Back to top

Comments

  • robots.txt for https://www.chewvalleytrees.co.uk/
  • live - don't allow web crawlers to index cpresources/ or vendor/
  • BingBot was indexing old URLs and hitting the server multiple times so this should stop that
Back to top