nap.nationalacademies.org
robots.txt

Robots Exclusion Standard data for nap.nationalacademies.org

Resource Scan

Scan Details

Site Domain nap.nationalacademies.org
Base Domain nationalacademies.org
Scan Status Ok
Last Scan2025-03-10T02:16:30+00:00
Next Scan 2025-04-09T02:16:30+00:00

Last Scan

Scanned2025-03-10T02:16:30+00:00
URL https://nap.nationalacademies.org/robots.txt
Domain IPs 144.171.1.3
Response IP 144.171.1.3
Found Yes
Hash 51c80c34b8f8ef77b0816b6ee00234ce6f564465005d513bc4aec4a7dae9c8c6
SimHash 54914914e693

Groups

*

Rule Path
Disallow /pdf/
Disallow /nap-cgi/
Disallow /cart/
Disallow /napsearch.php
Disallow /booksearch.php
Disallow /download.php
Disallow /login.php
Disallow /related.php
Disallow /mynap.php

googlebot

Rule Path
Disallow /pdf/
Disallow /nap-cgi/
Disallow /cart/
Disallow /napsearch.php
Disallow /booksearch.php
Disallow /download.php
Disallow /login.php
Disallow /related.php
Disallow /mynap.php

bingbot

Rule Path
Disallow /pdf/
Disallow /nap-cgi/
Disallow /cart/
Disallow /napsearch.php
Disallow /booksearch.php
Disallow /download.php
Disallow /login.php
Disallow /related.php
Disallow /mynap.php

msnbot

Rule Path
Disallow /pdf/
Disallow /nap-cgi/
Disallow /napsearch.php
Disallow /booksearch.php
Disallow /download.php
Disallow /login.php
Disallow /related.php
Disallow /mynap.php

charlotte

Rule Path
Disallow /cart/
Disallow /napsearch.php
Disallow /booksearch.php
Disallow /download.php
Disallow /login.php
Disallow /related.php
Disallow /mynap.php

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /