disciple.harvest.org
robots.txt

Robots Exclusion Standard data for disciple.harvest.org

Resource Scan

Scan Details

Site Domain disciple.harvest.org
Base Domain harvest.org
Scan Status Ok
Last Scan2025-09-20T04:35:39+00:00
Next Scan 2025-10-04T04:35:39+00:00

Last Scan

Scanned2025-09-20T04:35:39+00:00
URL https://disciple.harvest.org/robots.txt
Domain IPs 147.185.161.77, 147.185.161.78
Response IP 147.185.161.78
Found Yes
Hash d7554eff35af443c087c8ffd106814c21eaee2b870df99e9a3a64e8e9b809fc4
SimHash 6b0c8a12cbb1

Groups

botify
spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

*

Rule Path
Allow /$
Disallow /

Other Records

Field Value
sitemap https://disciple.harvest.org/sitemap.xml