Log In Sign Up

indexfungorum.org
robots.txt

Robots Exclusion Standard data for indexfungorum.org

Resource Scan

Scan Details

Site Domain indexfungorum.org
Base Domain indexfungorum.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-08-21T03:07:10+00:00
Next Scan 2025-09-20T03:07:10+00:00

Last Successful Scan

Scanned2025-06-29T22:39:19+00:00
URL https://indexfungorum.org/robots.txt
Domain IPs 213.48.60.82
Response IP 213.48.60.82
Found Yes
Hash 906a9cc2c4907e45165a84b565dbe9962c689e8c56561df6a3b93d4688b42f62
SimHash a4599511e36b

Groups

ahrefsbot
semrushbot
dataforseobot
zoombot
ru_bot
pinterestbot
rytebot
bytespider
screaming frog seo spider
sogou web spider
siteauditbot
keys-so-bot
yak
anthropic-ai
yisouspider
petalbot
serpstatbot
dotbot
yandexbot
aliyunsecbot
awariobot
farmerduckybot
adsbot-google-mobile

Rule Path
Disallow /

*

Rule Path
Disallow /IndexFungorumIndexToNames.xlsx
Disallow /IMAGES/dirlist/
Disallow /WebServiceHost/
Disallow /Names/captcha.asp
Disallow /Names/Names.asp?*
Disallow /Names/IndexFungorumAdd.asp
Disallow /Names/IndexFungorumAddInsertAddition.asp
Disallow /Names/IndexFungorumAddNameEmailer.asp
Disallow /Names/IndexFungorumAddNameValidation.asp
Disallow /Names/IndexFungorumPasswordRecover.asp
Disallow /Names/IndexFungorumPublishName.asp
Disallow /Names/IndexFungorumPublishNamePreselect.asp
Disallow /Names/IndexFungorumPublishNamePreview.asp
Disallow /Names/IndexFungorumRegisterName.asp
Disallow /Names/IndexFungorumRegisterNameCompletion.asp
Disallow /Names/IndexFungorumRegisterNameEmailer.asp
Disallow /Names/IndexFungorumRegisterUserHelp.htm
Disallow /Names/IndexFungorumRegisterUserActivation.asp
Disallow /Names/IndexFungorumUpdate.asp
Disallow /Names/IndexFungorumUpdateInsertUpdate.asp
Disallow /Names/IndexFungorumUpdateNameEmailer.asp
Disallow /Names/login.asp
Disallow /Names/login2.asp
Disallow /Names/Utility.asp

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • This robots.txt file guides the desired crawling of URLs under https://indexfungorum.org
  • Requesting user agents reduce crawling times, where possible
  • Disallowing pages that are not conducive to crawling, indexing and/or providing search results.
Back to top