chabad.edu
robots.txt
Robots Exclusion Standard data for chabad.edu
Resource Scan
Scan Details
| Site Domain | chabad.edu |
| Base Domain | chabad.edu |
| Scan Status | Ok |
| Last Scan | 2025-03-29T04:25:53+00:00 |
| Next Scan | 2025-04-28T04:25:53+00:00 |
Last Scan
| Scanned | 2025-03-29T04:25:53+00:00 |
| URL | https://www.chabad.edu/robots.txt |
| Domain IPs | 104.21.69.117, 172.67.207.254, 2606:4700:3031::ac43:cffe, 2606:4700:3033::6815:4575 |
| Response IP | 104.21.69.117 |
| Found | Yes |
| Hash | cf118efdebf80cada35ad79a88543a6fcd66d388399e6aaed71dc687b5cab9bd |
| SimHash | f885ded1cbb2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*print%3Dtrue |
| Disallow | /*print/true |
| Disallow | /dailystudy/ |
| Disallow | /Dailystudy/ |
| Disallow | /DailyStudy/ |
| Disallow | /holidays/ |
| Disallow | /Holidays/ |
| Disallow | /magazine/ |
| Disallow | /Magazine/ |
| Disallow | /parshah/ |
| Disallow | /Parshah/ |
| Disallow | /centers/ |
| Disallow | /Centers/ |
| Disallow | /multimedia/ |
| Disallow | /Multimedia/ |
| Disallow | /therebbe/ |
| Disallow | /Therebbe/ |
| Disallow | /TheRebbe/ |
| Disallow | /generic.asp |
| Disallow | /generic_cdo |
| Disallow | /calendar/ |
| Disallow | /Calendar/ |
| Disallow | /search/ |
| Disallow | /Search/ |
| Disallow | /scripts/ |
| Disallow | /global/about/ |
| Disallow | /special/ |
| Disallow | /Special/ |
| Disallow | /theJewishWoman/ |
| Disallow | /TheJewishWoman/ |
| Disallow | /kids/ |
| Disallow | /Kids/ |
| Disallow | /news/ |
| Disallow | /News/ |
| Disallow | /home/ |
| Disallow | /blogs/ |
| Disallow | /Blogs/ |
| Disallow | /tools/email.asp |
| Disallow | /tools/login/ |
| Disallow | /tools/rss/ |
| Disallow | /tools/donate/ |
| Disallow | /tools/subscribe/ |
| Disallow | /tools/unsubscribe/ |
| Disallow | /tools/cards/ |
| Disallow | /library/ |
| Disallow | /Library/ |
| Disallow | *tdate* |
| Disallow | *tDate* |
| Disallow | *TDate* |
| Disallow | /kabbalah/ |
| Disallow | /Kabbalah/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |