Log In Sign Up

heste-nettet.dk
robots.txt

Robots Exclusion Standard data for heste-nettet.dk

Resource Scan

Scan Details

Site Domain heste-nettet.dk
Base Domain heste-nettet.dk
Scan Status Ok
Last Scan2024-11-14T16:53:12+00:00
Next Scan 2024-11-21T16:53:12+00:00

Last Scan

Scanned2024-11-14T16:53:12+00:00
URL https://heste-nettet.dk/robots.txt
Redirect https://www.heste-nettet.dk/robots.txt
Redirect Domain www.heste-nettet.dk
Redirect Base heste-nettet.dk
Domain IPs 104.26.6.77, 104.26.7.77, 172.67.73.35, 2606:4700:20::681a:64d, 2606:4700:20::681a:74d, 2606:4700:20::ac43:4923
Redirect IPs 104.26.6.77, 104.26.7.77, 172.67.73.35, 2606:4700:20::681a:64d, 2606:4700:20::681a:74d, 2606:4700:20::ac43:4923
Response IP 104.26.7.77
Found Yes
Hash 968cc6908eda608499221bdaff1baad9c660ff478cf58ce7f1829895a57b0e4b
SimHash 6db0d877efd0

Groups

*

Rule Path
Disallow /admin/
Disallow /forum/11/
Disallow /forum//11/
Disallow /forum/23/
Disallow /forum//23/
Disallow /forum/add_to_mydebates/
Disallow /forum//add_to_mydebates/
Disallow /lodge/
Disallow /nyheder/tags.php
Disallow /nyheder//tags.php
Disallow /saml/

Other Records

Field Value
crawl-delay 1

revivebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /admin/
Disallow /forum/11/
Disallow /forum//11/
Disallow /forum/23/
Disallow /forum//23/
Disallow /forum/add_to_mydebates/
Disallow /forum//add_to_mydebates/
Disallow /lodge/
Disallow /nyheder/tags.php
Disallow /nyheder//tags.php
Disallow /saml/

Other Records

Field Value
crawl-delay 0
Back to top

Other Records

Field Value
sitemap https://www.heste-nettet.dk/sitemap.xml
Back to top

Comments

  • default
  • block unknown/attacking
  • dupplicate of default but with crawl-delay = 0
Back to top