lwf.org
robots.txt

Robots Exclusion Standard data for lwf.org

Resource Scan

Scan Details

Site Domain lwf.org
Base Domain lwf.org
Scan Status Ok
Last Scan5/11/2025, 5:13:27 PM
Next Scan 5/18/2025, 5:13:27 PM

Last Scan

Scanned5/11/2025, 5:13:27 PM
URL https://lwf.org/robots.txt
Domain IPs 104.21.2.164, 172.67.129.100, 2606:4700:3031::6815:2a4, 2606:4700:3032::ac43:8164
Response IP 104.21.2.164
Found Yes
Hash 29234d52f81e6c257f2f95a99c11e9fbbc176de71a3f05ed5c1f9d73d582b99b
SimHash 4a501d523fb3

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /*?p=actions%2F*

Other Records

Field Value
sitemap https://www.lwf.org/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.lwf.org/
  • live - don't allow web crawlers to index cpresources/ or vendor/