wiki.dolphin-emu.org
robots.txt

Robots Exclusion Standard data for wiki.dolphin-emu.org

Resource Scan

Scan Details

Site Domain wiki.dolphin-emu.org
Base Domain dolphin-emu.org
Scan Status Ok
Last Scan2025-10-30T04:32:22+00:00
Next Scan 2025-11-29T04:32:22+00:00

Last Scan

Scanned2025-10-30T04:32:22+00:00
URL https://wiki.dolphin-emu.org/robots.txt
Domain IPs 185.31.40.20, 2a00:b6e0:1:20:11::1
Response IP 185.31.40.20
Found Yes
Hash 3c006950ea78b99c920ebce396597e6c60860bd10993937385e6311873556c19
SimHash 74fe4b91c5e6

Groups

*

Rule Path
Disallow /index.php?diff=
Disallow /index.php?oldid=
Disallow /index.php?title=Help
Disallow /index.php?title=Image
Disallow /index.php?title=MediaWiki
Disallow /index.php?title=Special%3A
Disallow /index.php?title=Template
Disallow /skins/
Disallow /index.php?*&action=
Disallow /index.php?*&oldid=
Disallow /index.php?*&diff=
Disallow /index.php?*&title=Special%3A*
Disallow /index.php?*&title=Template%3A*

ai2bot
ai2bot-dolma
aihitbot
amazonbot
anthropic-ai
applebot-extended
brightbot 1.0
bytespider
ccbot
claudebot
cohere-ai
cohere-training-data-crawler
cotoyogi
crawlspace
diffbot
duckassistbot
facebookbot
factset_spyderbot
firecrawlagent
friendlycrawler
google-extended
googleother
googleother-image
googleother-video
gptbot
iaskspider/2.0
icc-crawler
imagesiftbot
img2dataset
imgproxy
isscyberriskcrawler
kangaroo bot
meta-externalagent
meta-externalfetcher
novaact
omgili
omgilibot
operator
pangubot
perplexitybot
petalbot
scrapy
semrushbot-ocob
semrushbot-swa
sidetrade indexer bot
tiktokspider
timpibot
velenpublicwebcrawler
webzio-extended
youbot

Rule Path
Disallow /

Comments

  • https://www.mediawiki.org/wiki/Manual:Robots.txt
  • Additional wildcard-based rules in case these query parameters come at the end
  • Block AI Crawlers
  • v1.29: https://github.com/ai-robots-txt/ai.robots.txt
  • Modified to unblock bots triggered by user action: ChatGPT-User, Claude-Web, OAI-SearchBot, Perplexity-User, Applebot