physiciandirectory.brighamandwomens.org
robots.txt
Robots Exclusion Standard data for physiciandirectory.brighamandwomens.org
Resource Scan
Scan Details
| Site Domain | physiciandirectory.brighamandwomens.org |
| Base Domain | brighamandwomens.org |
| Scan Status | Ok |
| Last Scan | 2024-11-03T10:41:50+00:00 |
| Next Scan | 2024-12-03T10:41:50+00:00 |
Last Scan
| Scanned | 2024-11-03T10:41:50+00:00 |
| URL | https://physiciandirectory.brighamandwomens.org/robots.txt |
| Domain IPs | 18.210.198.20 |
| Response IP | 18.210.198.20 |
| Found | Yes |
| Hash | 5a3e7b00d44867111e46c09fb0d82d1b008bab1967cfe67366cedcc3cb2fc978 |
| SimHash | 6d565891c7b0 |
Groups
bingbot
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
googlebot
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
googlebot-mobile
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
gsa-crawler
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
ia_archiver
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
idol_bot
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
psbot
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
sitecheck-sitecrawl by siteimprove.com
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
slurp
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
swiftbot
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
teoma
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
yahoo! slurp
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
yahoo-mmcrawler
| Rule | Path |
|---|---|
| Disallow | /pdfdownload |
| Disallow | /PDFDownload |
| Disallow | /shared-content |
| Disallow | /Error |
| Disallow | /error |
| Disallow | /vCard |
| Disallow | /PDF_Faulkner |
| Disallow | /pdf_faulkner |
| Disallow | /Internal |
| Disallow | /internal |
| Disallow | /PDF_Default |
| Disallow | /pdf_default |
| Disallow | /listings |
| Disallow | /listings |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://physiciandirectory.brighamandwomens.org/sitemap.xml |