gaeludyr.is
robots.txt

Robots Exclusion Standard data for gaeludyr.is

Resource Scan

Scan Details

Site Domain gaeludyr.is
Base Domain gaeludyr.is
Scan Status Ok
Last Scan2024-09-04T16:58:25+00:00
Next Scan 2024-10-04T16:58:25+00:00

Last Scan

Scanned2024-09-04T16:58:25+00:00
URL https://gaeludyr.is/robots.txt
Redirect https://www.gaeludyr.is/robots.txt
Redirect Domain www.gaeludyr.is
Redirect Base gaeludyr.is
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.142, 76.76.21.9
Response IP 76.76.21.164
Found Yes
Hash d0c2111a03666391fc94d4ea5d9acc5115a2b6c9fda95f5fa77a76b3a9a323a2
SimHash 500c045357b0

Groups

ccbot

Rule Path
Disallow /

buck

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

*

Rule Path
Allow /*
Disallow /api/*

Other Records

Field Value
sitemap https://www.gaeludyr.is/sitemap.xml

Warnings

  • `host` is not a known field.