arhag.co.uk
robots.txt

Robots Exclusion Standard data for arhag.co.uk

Resource Scan

Scan Details

Site Domain arhag.co.uk
Base Domain arhag.co.uk
Scan Status Ok
Last Scan5/23/2025, 4:33:11 AM
Next Scan 5/30/2025, 4:33:11 AM

Last Scan

Scanned5/23/2025, 4:33:11 AM
URL https://arhag.co.uk/robots.txt
Redirect https://www.arhag.co.uk/robots.txt
Redirect Domain www.arhag.co.uk
Redirect Base arhag.co.uk
Domain IPs 143.42.254.148
Redirect IPs 23.209.46.76, 23.209.46.96, 2600:1413:b000:1e::17d1:2e4c, 2600:1413:b000:1e::17d1:2e60
Response IP 23.45.207.176
Found Yes
Hash 397bdd9d774393a519302b9fed7dbc99a351b1917dd2a25611f12f4aafe08610
SimHash c1781d563792

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://www.arhag.co.uk/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.arhag.co.uk/
  • live - don't allow web crawlers to index cpresources/ or vendor/