nia.nih.gov
robots.txt
Robots Exclusion Standard data for nia.nih.gov
Resource Scan
Scan Details
Site Domain | nia.nih.gov |
Base Domain | nih.gov |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-04-21T18:21:01+00:00 |
Next Scan | 2024-05-21T18:21:01+00:00 |
Last Successful Scan
Scanned | 2024-02-29T18:20:17+00:00 |
URL | https://nia.nih.gov/robots.txt |
Redirect | https://www.nia.nih.gov/robots.txt |
Redirect Domain | www.nia.nih.gov |
Redirect Base | nih.gov |
Domain IPs | 76.223.47.21 |
Redirect IPs | 13.33.33.33, 13.33.33.43, 13.33.33.72, 13.33.33.74, 2600:9000:229f:1400:16:fb03:50c0:93a1, 2600:9000:229f:200:16:fb03:50c0:93a1, 2600:9000:229f:400:16:fb03:50c0:93a1, 2600:9000:229f:4400:16:fb03:50c0:93a1, 2600:9000:229f:7c00:16:fb03:50c0:93a1, 2600:9000:229f:c800:16:fb03:50c0:93a1, 2600:9000:229f:d000:16:fb03:50c0:93a1, 2600:9000:229f:f000:16:fb03:50c0:93a1 |
Response IP | 13.33.33.74 |
Found | Yes |
Hash | fb481f357693be93505a7e1cb562728f40eb4c233eaab4ba052dfed96ba4e677 |
SimHash | 3916bd51c760 |
Groups
*
Rule | Path |
---|---|
Allow | /core/*.css$ |
Allow | /core/*.css? |
Allow | /core/*.js$ |
Allow | /core/*.js? |
Allow | /core/*.gif |
Allow | /core/*.jpg |
Allow | /core/*.jpeg |
Allow | /core/*.png |
Allow | /core/*.svg |
Allow | /profiles/*.css$ |
Allow | /profiles/*.css? |
Allow | /profiles/*.js$ |
Allow | /profiles/*.js? |
Allow | /profiles/*.gif |
Allow | /profiles/*.jpg |
Allow | /profiles/*.jpeg |
Allow | /profiles/*.png |
Allow | /profiles/*.svg |
Disallow | /core/ |
Disallow | /profiles/ |
Disallow | /README.txt |
Disallow | /web.config |
Disallow | /admin/ |
Disallow | /comment/reply/ |
Disallow | /filter/tips |
Disallow | /node/add/ |
Disallow | /search/ |
Disallow | /user/register |
Disallow | /user/password |
Disallow | /user/login |
Disallow | /user/logout |
Disallow | /index.php/admin/ |
Disallow | /index.php/comment/reply/ |
Disallow | /index.php/filter/tips |
Disallow | /index.php/node/add/ |
Disallow | /index.php/search/ |
Disallow | /index.php/user/password |
Disallow | /index.php/user/register |
Disallow | /index.php/user/login |
Disallow | /index.php/user/logout |
Disallow | /taxonomy |
Disallow | /site-search |
Disallow | /node* |
Disallow | /node |
Comments