giant.health
robots.txt

Robots Exclusion Standard data for giant.health

Resource Scan

Scan Details

Site Domain giant.health
Base Domain giant.health
Scan Status Ok
Last Scan2024-09-18T12:03:31+00:00
Next Scan 2024-10-18T12:03:31+00:00

Last Scan

Scanned2024-09-18T12:03:31+00:00
URL https://giant.health/robots.txt
Redirect https://www.giant.health/robots.txt
Redirect Domain www.giant.health
Redirect Base giant.health
Domain IPs 13.227.254.123, 13.227.254.3, 13.227.254.37, 13.227.254.8
Redirect IPs 13.227.254.123, 13.227.254.3, 13.227.254.37, 13.227.254.8
Response IP 13.227.254.37
Found Yes
Hash 95e40a018f1e76f2d0c638ed38ec78b0689ddd71d09974da6a36a60614cb4aed
SimHash c80cd82485f1

Groups

*
google
googlebot
twitterbot

Rule Path
Allow /
Disallow
Disallow /js
Disallow /cache
Disallow /files
Disallow /admin

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.giant.health/sitemap.xml