intotheglacier.is
robots.txt

Robots Exclusion Standard data for intotheglacier.is

Resource Scan

Scan Details

Site Domain intotheglacier.is
Base Domain intotheglacier.is
Scan Status Ok
Last Scan2024-05-09T10:36:29+00:00
Next Scan 2024-06-08T10:36:29+00:00

Last Scan

Scanned2024-05-09T10:36:29+00:00
URL https://intotheglacier.is/robots.txt
Domain IPs 172.66.40.85, 172.66.43.171, 2606:4700:3108::ac42:2855, 2606:4700:3108::ac42:2bab
Response IP 172.66.40.85
Found Yes
Hash f7aa888fe10caaf04b66ad66773065666967a6e62e66c19bd93841f7016f8f38
SimHash c008de128213

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Disallow /cart/
Disallow /booking-details/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://intotheglacier.is/sitemap.xml