Log In Sign Up

maps.mla.org
robots.txt

Robots Exclusion Standard data for maps.mla.org

Resource Scan

Scan Details

Site Domain maps.mla.org
Base Domain mla.org
Scan Status Ok
Last Scan2024-05-18T21:04:43+00:00
Next Scan 2024-06-17T21:04:43+00:00

Last Scan

Scanned2024-05-18T21:04:43+00:00
URL https://maps.mla.org/robots.txt
Redirect https://www.maps.mla.org/robots.txt
Redirect Domain www.maps.mla.org
Redirect Base mla.org
Domain IPs 13.227.254.10, 13.227.254.128, 13.227.254.29, 13.227.254.66, 2600:9000:200a:1600:19:2a30:1480:93a1, 2600:9000:200a:3000:19:2a30:1480:93a1, 2600:9000:200a:5800:19:2a30:1480:93a1, 2600:9000:200a:6e00:19:2a30:1480:93a1, 2600:9000:200a:7800:19:2a30:1480:93a1, 2600:9000:200a:d600:19:2a30:1480:93a1, 2600:9000:200a:e400:19:2a30:1480:93a1, 2600:9000:200a:f400:19:2a30:1480:93a1
Redirect IPs 13.227.254.10, 13.227.254.128, 13.227.254.29, 13.227.254.66, 2600:9000:200a:2200:19:2a30:1480:93a1, 2600:9000:200a:6600:19:2a30:1480:93a1, 2600:9000:200a:7200:19:2a30:1480:93a1, 2600:9000:200a:8000:19:2a30:1480:93a1, 2600:9000:200a:9c00:19:2a30:1480:93a1, 2600:9000:200a:aa00:19:2a30:1480:93a1, 2600:9000:200a:e200:19:2a30:1480:93a1, 2600:9000:200a:ee00:19:2a30:1480:93a1
Response IP 13.227.254.128
Found Yes
Hash ec1ab0151477b95524e04a5b83b471418e5c2d681eb633df681119b167f25d26
SimHash 681151c1c0f1

Groups

*

Rule Path
Disallow /

googlebot
bingbot
bingpreview
msnbot
slurp
duckduckbot
applebot
ia_archiver
facebookexternalhit
twitterbot
linkedinbot

Rule Path
Allow /
Disallow /admin/
Disallow /site_admin/
Disallow /search
Disallow /search/
Disallow /content/search
Disallow /content/advancedsearch
Disallow /content/tipafriend
Disallow /layout/set/print
Disallow /rss
Disallow /media/
Disallow /ezinfo/
Disallow /user/
Disallow /test-area/
Disallow /bulletin/search/
Disallow /shop/search/
Back to top

Comments

  • Disallow all
  • But allow only important bots
Back to top