famsf.org
robots.txt

Robots Exclusion Standard data for famsf.org

Resource Scan

Scan Details

Site Domain famsf.org
Base Domain famsf.org
Scan Status Ok
Last Scan2025-05-18T09:39:13+00:00
Next Scan 2025-06-17T09:39:13+00:00

Last Scan

Scanned2025-05-18T09:39:13+00:00
URL https://famsf.org/robots.txt
Redirect https://www.famsf.org/robots.txt
Redirect Domain www.famsf.org
Redirect Base famsf.org
Domain IPs 151.101.130.216, 151.101.194.216, 151.101.2.216, 151.101.66.216
Redirect IPs 151.101.130.216, 151.101.194.216, 151.101.2.216, 151.101.66.216
Response IP 151.101.2.216
Found Yes
Hash 9ee2f1c63f836c13f4d2cb567f32fd4ceccd8b340d688b0a3d049884c7ef7ae3
SimHash 624cc800e300

Groups

*

Rule Path
Disallow /art/*
Disallow /about/rent-the-museums/rental-event-inquiry-form
Disallow /about/rent-the-museums/confirmation

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

viberbot

Rule Path
Disallow /

yak

Rule Path
Disallow /

yandex

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10