preshomes.org
robots.txt

Robots Exclusion Standard data for preshomes.org

Resource Scan

Scan Details

Site Domain preshomes.org
Base Domain preshomes.org
Scan Status Ok
Last Scan2025-10-31T18:37:19+00:00
Next Scan 2025-11-30T18:37:19+00:00

Last Scan

Scanned2025-10-31T18:37:19+00:00
URL https://preshomes.org/robots.txt
Domain IPs 104.26.12.6, 104.26.13.6, 172.67.75.139, 2606:4700:20::681a:c06, 2606:4700:20::681a:d06, 2606:4700:20::ac43:4b8b
Response IP 172.67.75.139
Found Yes
Hash 26fddca126c9e156045cf68de896a0fc4d3f65d8a7c52cc29b967966ae5d4698
SimHash 405cd0c1b600

Groups

googlebot

Rule Path
Disallow

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

amazonbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /do-not-crawl/

claudebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /