Log In Sign Up

aaep.org
robots.txt

Robots Exclusion Standard data for aaep.org

Resource Scan

Scan Details

Site Domain aaep.org
Base Domain aaep.org
Scan Status Ok
Last Scan2026-01-04T03:23:04+00:00
Next Scan 2026-02-03T03:23:04+00:00

Last Scan

Scanned2026-01-04T03:23:04+00:00
URL https://aaep.org/robots.txt
Domain IPs 162.159.134.42
Response IP 162.159.134.42
Found Yes
Hash ba8f0a209614be85a49f0333e5a0fa05e1ea927d5a8cf3837e28e2e467687e6f
SimHash 5528cc01e3f2

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

facebookexternalhit

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php

facebot

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php

*

Rule Path
Disallow
Back to top

Comments

  • Limit Bingbot crawl rate
  • Limit Amazonbot crawl rate
  • Limit Facebook crawler
  • Allow all other bots
Back to top