annonse.aller.no
robots.txt

Robots Exclusion Standard data for annonse.aller.no

Resource Scan

Scan Details

Site Domain annonse.aller.no
Base Domain aller.no
Scan Status Ok
Last Scan2024-05-20T15:19:56+00:00
Next Scan 2024-06-03T15:19:56+00:00

Last Scan

Scanned2024-05-20T15:19:56+00:00
URL https://annonse.aller.no/robots.txt
Domain IPs 104.18.4.170, 104.18.5.170, 2606:4700::6812:4aa, 2606:4700::6812:5aa
Response IP 104.18.5.170
Found Yes
Hash 0aa2e659e37502b350e9f9bdddc27bb7817fd5832b8fdf5a48fa10f5bbeee572
SimHash 41381d563f93

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://annonse.aller.no/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://annonse.aller.no/
  • live - don't allow web crawlers to index cpresources/ or vendor/