nlm.nih.gov
robots.txt
Robots Exclusion Standard data for nlm.nih.gov
Resource Scan
Scan Details
| Site Domain | nlm.nih.gov |
| Base Domain | nih.gov |
| Scan Status | Ok |
| Last Scan | 2024-04-22T14:29:36+00:00 |
| Next Scan | 2024-05-22T14:29:36+00:00 |
Last Scan
| Scanned | 2024-04-22T14:29:36+00:00 |
| URL | https://nlm.nih.gov/robots.txt |
| Redirect | https://www.nlm.nih.gov/robots.txt |
| Redirect Domain | www.nlm.nih.gov |
| Redirect Base | nih.gov |
| Domain IPs | 130.14.16.110, 2607:f220:41e:1016::110 |
| Redirect IPs | 2600:9000:2364:1200:18:ab46:4b40:93a1, 2600:9000:2364:1e00:18:ab46:4b40:93a1, 2600:9000:2364:2600:18:ab46:4b40:93a1, 2600:9000:2364:4600:18:ab46:4b40:93a1, 2600:9000:2364:5400:18:ab46:4b40:93a1, 2600:9000:2364:6000:18:ab46:4b40:93a1, 2600:9000:2364:7a00:18:ab46:4b40:93a1, 2600:9000:2364:c200:18:ab46:4b40:93a1, 3.163.24.118, 3.163.24.128, 3.163.24.26, 3.163.24.51 |
| Response IP | 18.165.171.21 |
| Found | Yes |
| Hash | 20c2103546b1780b87b40181394fca5d3845d6f4f580d8105bbd8c5cc2d48325 |
| SimHash | a019cd7e7495 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /accenture/ |
| Disallow | /cgi/ |
| Disallow | /sitemessages/ |
| Disallow | /admin/ |
| Disallow | /homedev/ |
| Disallow | /test/ |
| Disallow | /web/documentation/accessibility-checklists/ |
| Disallow | /archive/ |
| Disallow | /medlineplus2/ |
| Disallow | /medlineplus/medplapp/ |
| Disallow | /medlineplus/URLchecker/ |
| Disallow | /medlineplus/usp/ |
| Disallow | /medlineplus/logs/ |
| Disallow | /medlineplus/stats/ |
| Disallow | /medlineplus/glftp/ |
| Disallow | /medlineplus/procedures/ |
| Disallow | /medlineplus/mobilefeeds/ |
| Disallow | /medlineplus/spanish/mobilefeeds/ |
| Disallow | /medlineplus/feeds/topics/ |
| Disallow | /medlineplus/groupfeeds/ |
| Disallow | /medlineplus/spanish/feeds/topics/ |
| Disallow | /medlineplus/spanish/groupfeeds/ |
| Disallow | /medlineplus/golocal/ |
| Disallow | /medlineplus/golocaldocs/ |
| Disallow | /medlineplus/xml/ |
| Disallow | /medlineplus/xml/vocabulary/ |
| Disallow | /medlineplus/scwidget/topsearch.html |
| Disallow | /medlineplus/spanish/scwidget/topsearch.html |
| Disallow | /research/visible/vhpconf98/AUTHORS/WERNER2/ |
| Disallow | /research/visible/vhpconf98/AUTHORS/SACHSE2/ |
| Disallow | /golocal/extranet/ |
Comments