herbalgram.org
robots.txt

Robots Exclusion Standard data for herbalgram.org

Resource Scan

Scan Details

Site Domain herbalgram.org
Base Domain herbalgram.org
Scan Status Ok
Last Scan2024-05-04T11:16:27+00:00
Next Scan 2024-06-03T11:16:27+00:00

Last Scan

Scanned2024-05-04T11:16:27+00:00
URL https://herbalgram.org/robots.txt
Redirect https://www.herbalgram.org/robots.txt
Redirect Domain www.herbalgram.org
Redirect Base herbalgram.org
Domain IPs 172.203.232.179
Redirect IPs 172.203.232.179
Response IP 172.203.232.179
Found Yes
Hash e1e482cec1e5f19c644dd5394aa6c99f33c065918906605589cd8e169a034cf6
SimHash e9089a72c331

Groups

addsearchbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 20

siteauditbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

semrushbot-si

Rule Path
Allow /

*

Rule Path
Disallow