bths.edu
robots.txt
Robots Exclusion Standard data for bths.edu
Resource Scan
Scan Details
Site Domain | bths.edu |
Base Domain | bths.edu |
Scan Status | Ok |
Last Scan | 2024-08-29T15:49:03+00:00 |
Next Scan | 2024-09-28T15:49:03+00:00 |
Last Scan
Scanned | 2024-08-29T15:49:03+00:00 |
URL | https://bths.edu/robots.txt |
Redirect | https://www.bths.edu/servlet/RobotsFileService |
Redirect Domain | www.bths.edu |
Redirect Base | bths.edu |
Domain IPs | 34.85.210.27 |
Redirect IPs | 34.85.210.27 |
Response IP | 34.85.210.27 |
Found | Yes |
Hash | d9fff455132b25f93978a9c244d72e3a9c73db14e747e676ee5c910770918bcd |
SimHash | 73dd9d510e39 |
Groups
googlebot
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
Disallow | /m/classes |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
googlebot-mobile
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
Disallow | /m/classes |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
twitterbot
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
Disallow | /m/classes |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
w3c-checklink
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/ |
Disallow | /m/classes/ |
Disallow | /ourpages |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
bingbot
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
Disallow | /m/classes |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
msnbot
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
Disallow | /m/classes |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
accessiblewebbot
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
facebookexternalhit
Rule | Path |
---|---|
Disallow | /apps/ajax/ |
Disallow | /apps/classes/ |
Disallow | /apps/directory/ |
Disallow | /apps/events/events_rss.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/view_calendar.jsp |
Disallow | /apps/events/list_pff.jsp |
Disallow | /apps/events/calendar_yearly_pff.jsp |
Disallow | /apps/events/calendar_pff.jsp |
Disallow | /apps/events/calendar_month_pff.jsp |
Disallow | /apps/email |
Disallow | /apps/email/ |
Disallow | /apps/email/index.jsp |
Disallow | /apps/file_manager/ |
Disallow | /apps/mediaplayer-3-16/ |
Disallow | /apps/ourpages/ |
Disallow | /apps/ourpages |
Disallow | /apps/forms/ |
Disallow | /apps/login/ |
Disallow | /apps/s/ |
Disallow | /apps/thickbox/ |
Disallow | /apps/tools/ |
Disallow | /apps/videos/ |
Disallow | /apps/cross.jsp |
Disallow | /apps/temp.jsp |
Disallow | /m/api |
Disallow | /m/classes |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
*
Rule | Path |
---|---|
Disallow | / |
Allow | /robots.txt |
Other Records
Field | Value |
---|---|
sitemap | https://www.bths.edu/sitemap.xml |