charliewaller.org
robots.txt

Robots Exclusion Standard data for charliewaller.org

Resource Scan

Scan Details

Site Domain charliewaller.org
Base Domain charliewaller.org
Scan Status Ok
Last Scan2025-09-14T18:30:53+00:00
Next Scan 2025-10-14T18:30:53+00:00

Last Scan

Scanned2025-09-14T18:30:53+00:00
URL https://charliewaller.org/robots.txt
Domain IPs 104.19.191.28
Response IP 104.19.191.28
Found Yes
Hash 28907da2f0206871f61b2e870a40225ecfeb68049224d3426d56af27fd49cd86
SimHash 6f1499436724

Groups

*

Rule Path
Disallow /umbraco/
Disallow /umbraco_client/
Disallow /App_Plugins/
Disallow /App_Data/
Disallow /config/
Allow /

Other Records

Field Value
sitemap https://charliewaller.org/sitemap.xml

Comments

  • Allow all well-behaved crawlers unless otherwise specified
  • Prevent indexing of Umbraco back-office and system folders
  • Let crawlers access everything else
  • Point to your XML sitemaps