mainweb.awsprod.nlm.nih.gov
robots.txt
Robots Exclusion Standard data for mainweb.awsprod.nlm.nih.gov
Resource Scan
Scan Details
Site Domain | mainweb.awsprod.nlm.nih.gov |
Base Domain | nih.gov |
Scan Status | Ok |
Last Scan | 2024-04-24T21:34:05+00:00 |
Next Scan | 2024-05-24T21:34:05+00:00 |
Last Scan
Scanned | 2024-04-24T21:34:05+00:00 |
URL | https://mainweb.awsprod.nlm.nih.gov/robots.txt |
Domain IPs | 13.226.2.31, 13.226.2.58, 13.226.2.59, 13.226.2.66, 2600:9000:20a6:1600:18:ab46:4b40:93a1, 2600:9000:20a6:2a00:18:ab46:4b40:93a1, 2600:9000:20a6:3e00:18:ab46:4b40:93a1, 2600:9000:20a6:a000:18:ab46:4b40:93a1, 2600:9000:20a6:a600:18:ab46:4b40:93a1, 2600:9000:20a6:b400:18:ab46:4b40:93a1, 2600:9000:20a6:d200:18:ab46:4b40:93a1, 2600:9000:20a6:dc00:18:ab46:4b40:93a1 |
Response IP | 18.165.171.117 |
Found | Yes |
Hash | 20c2103546b1780b87b40181394fca5d3845d6f4f580d8105bbd8c5cc2d48325 |
SimHash | a019cd7e7495 |
Groups
*
Rule | Path |
---|---|
Disallow | /accenture/ |
Disallow | /cgi/ |
Disallow | /sitemessages/ |
Disallow | /admin/ |
Disallow | /homedev/ |
Disallow | /test/ |
Disallow | /web/documentation/accessibility-checklists/ |
Disallow | /archive/ |
Disallow | /medlineplus2/ |
Disallow | /medlineplus/medplapp/ |
Disallow | /medlineplus/URLchecker/ |
Disallow | /medlineplus/usp/ |
Disallow | /medlineplus/logs/ |
Disallow | /medlineplus/stats/ |
Disallow | /medlineplus/glftp/ |
Disallow | /medlineplus/procedures/ |
Disallow | /medlineplus/mobilefeeds/ |
Disallow | /medlineplus/spanish/mobilefeeds/ |
Disallow | /medlineplus/feeds/topics/ |
Disallow | /medlineplus/groupfeeds/ |
Disallow | /medlineplus/spanish/feeds/topics/ |
Disallow | /medlineplus/spanish/groupfeeds/ |
Disallow | /medlineplus/golocal/ |
Disallow | /medlineplus/golocaldocs/ |
Disallow | /medlineplus/xml/ |
Disallow | /medlineplus/xml/vocabulary/ |
Disallow | /medlineplus/scwidget/topsearch.html |
Disallow | /medlineplus/spanish/scwidget/topsearch.html |
Disallow | /research/visible/vhpconf98/AUTHORS/WERNER2/ |
Disallow | /research/visible/vhpconf98/AUTHORS/SACHSE2/ |
Disallow | /golocal/extranet/ |
Comments