nde-ed.org
robots.txt

Robots Exclusion Standard data for nde-ed.org

Resource Scan

Scan Details

Site Domain nde-ed.org
Base Domain nde-ed.org
Scan Status Ok
Last Scan2025-09-06T09:28:08+00:00
Next Scan 2025-10-06T09:28:08+00:00

Last Scan

Scanned2025-09-06T09:28:08+00:00
URL https://nde-ed.org/robots.txt
Redirect https://www.nde-ed.org/robots.txt
Redirect Domain www.nde-ed.org
Redirect Base nde-ed.org
Domain IPs 20.241.39.52
Redirect IPs 20.241.39.52
Response IP 20.241.39.52
Found Yes
Hash 4c34bbfadf2eb6fe1b42df7a1182e2c43315d0ba64247aaa0fdeeb55981d0ed9
SimHash 3a20e1538f16

Groups

*

Rule Path
Disallow /*?*
Disallow */js_apps/*html
Disallow */Applet*html

Comments

  • Disallow all pages with query strings -- in particular the ?continuous_scroll=1 and the ?autoglossary=1 url's
  • Disallow direct applet indexing.