aeen.org
robots.txt

Robots Exclusion Standard data for aeen.org

Resource Scan

Scan Details

Site Domain aeen.org
Base Domain aeen.org
Scan Status Ok
Last Scan2025-10-03T00:57:04+00:00
Next Scan 2025-11-02T00:57:04+00:00

Last Scan

Scanned2025-10-03T00:57:04+00:00
URL https://aeen.org/robots.txt
Domain IPs 46.231.127.118
Response IP 46.231.127.118
Found Yes
Hash d1b3e62d1c1f3a761a200b58d6d552e5dc1985b81ef01b4fe5c60288d538604c
SimHash 4e54584ac681

Groups

*

Rule Path
Disallow /feed/
Disallow /trackback/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /page/
Disallow /xmlrpc.php

googlebot

Rule Path
Allow /*.css$
Allow /*.js$

mj12bot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

teoma

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

scrubby

Rule Path
Disallow /

robozilla
blexbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

psbot

Rule Path
Disallow /

asterias

Rule Path
Disallow /

yandex

Rule Path
Disallow /