gemstrends.com
robots.txt

Robots Exclusion Standard data for gemstrends.com

Resource Scan

Scan Details

Site Domain gemstrends.com
Base Domain gemstrends.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-09-26T15:58:59+00:00
Next Scan 2025-10-26T15:58:59+00:00

Last Successful Scan

Scanned2025-08-05T10:50:10+00:00
URL https://gemstrends.com/robots.txt
Domain IPs 104.21.49.236, 172.67.195.210, 2606:4700:3037::6815:31ec, 2606:4700:3037::ac43:c3d2
Response IP 172.67.195.210
Found Yes
Hash d6ac886d2e38c8c3c622020ea97ddc5c008ba1953b7126e9b0d12b54706268cd
SimHash 6851d9915357

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow /