botany.org
robots.txt

Robots Exclusion Standard data for botany.org

Resource Scan

Scan Details

Site Domain botany.org
Base Domain botany.org
Scan Status Ok
Last Scan2025-08-17T19:33:30+00:00
Next Scan 2025-09-16T19:33:30+00:00

Last Scan

Scanned2025-08-17T19:33:30+00:00
URL https://botany.org/robots.txt
Domain IPs 34.203.91.214
Response IP 34.203.91.214
Found Yes
Hash 019b979e5393f08216ee1984d57550ebabcd361dbe2427571f1889e747309f48
SimHash 007c9302fa80

Groups

linkpadbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot/5.1

Rule Path
Disallow /

semrushbot/1.1~bl

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

search.marginalia.nu

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /

bytespider

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

*

Rule Path
Disallow /config
Disallow /docs
Disallow /images
Disallow /install
Disallow /javascript
Disallow /lib
Disallow /locale
Disallow /modules
Disallow /plugins
Disallow /system
Disallow /themes
Disallow /ztemp
Disallow /admin.php
Disallow /ajax.php
Disallow /install.php
Disallow /jcss.php
Disallow /mo2json.php
Disallow /upgrade.php
Disallow /user.php