nlm.gov
robots.txt
Robots Exclusion Standard data for nlm.gov
Resource Scan
Scan Details
Site Domain | nlm.gov |
Base Domain | nlm.gov |
Scan Status | Ok |
Last Scan | 2024-06-01T18:46:08+00:00 |
Next Scan | 2024-07-01T18:46:08+00:00 |
Last Scan
Scanned | 2024-06-01T18:46:08+00:00 |
URL | https://nlm.gov/robots.txt |
Redirect | https://www.nlm.nih.gov/robots.txt |
Redirect Domain | www.nlm.nih.gov |
Redirect Base | nih.gov |
Domain IPs | 130.14.16.110, 2607:f220:41e:1016::110 |
Redirect IPs | 18.65.25.58, 18.65.25.60, 18.65.25.72, 18.65.25.99, 2600:9000:20a6:3600:18:ab46:4b40:93a1, 2600:9000:20a6:3800:18:ab46:4b40:93a1, 2600:9000:20a6:5600:18:ab46:4b40:93a1, 2600:9000:20a6:7c00:18:ab46:4b40:93a1, 2600:9000:20a6:9c00:18:ab46:4b40:93a1, 2600:9000:20a6:b800:18:ab46:4b40:93a1, 2600:9000:20a6:c600:18:ab46:4b40:93a1, 2600:9000:20a6:ca00:18:ab46:4b40:93a1 |
Response IP | 18.165.171.2 |
Found | Yes |
Hash | 20c2103546b1780b87b40181394fca5d3845d6f4f580d8105bbd8c5cc2d48325 |
SimHash | a019cd7e7495 |
Groups
*
Rule | Path |
---|---|
Disallow | /accenture/ |
Disallow | /cgi/ |
Disallow | /sitemessages/ |
Disallow | /admin/ |
Disallow | /homedev/ |
Disallow | /test/ |
Disallow | /web/documentation/accessibility-checklists/ |
Disallow | /archive/ |
Disallow | /medlineplus2/ |
Disallow | /medlineplus/medplapp/ |
Disallow | /medlineplus/URLchecker/ |
Disallow | /medlineplus/usp/ |
Disallow | /medlineplus/logs/ |
Disallow | /medlineplus/stats/ |
Disallow | /medlineplus/glftp/ |
Disallow | /medlineplus/procedures/ |
Disallow | /medlineplus/mobilefeeds/ |
Disallow | /medlineplus/spanish/mobilefeeds/ |
Disallow | /medlineplus/feeds/topics/ |
Disallow | /medlineplus/groupfeeds/ |
Disallow | /medlineplus/spanish/feeds/topics/ |
Disallow | /medlineplus/spanish/groupfeeds/ |
Disallow | /medlineplus/golocal/ |
Disallow | /medlineplus/golocaldocs/ |
Disallow | /medlineplus/xml/ |
Disallow | /medlineplus/xml/vocabulary/ |
Disallow | /medlineplus/scwidget/topsearch.html |
Disallow | /medlineplus/spanish/scwidget/topsearch.html |
Disallow | /research/visible/vhpconf98/AUTHORS/WERNER2/ |
Disallow | /research/visible/vhpconf98/AUTHORS/SACHSE2/ |
Disallow | /golocal/extranet/ |
Comments