runescape.wiki
robots.txt

Robots Exclusion Standard data for runescape.wiki

Resource Scan

Scan Details

Site Domain runescape.wiki
Base Domain runescape.wiki
Scan Status Ok
Last Scan2024-09-27T07:22:27+00:00
Next Scan 2024-10-04T07:22:27+00:00

Last Scan

Scanned2024-09-27T07:22:27+00:00
URL https://runescape.wiki/robots.txt
Domain IPs 104.22.60.210, 104.22.61.210, 172.67.13.152, 2606:4700:10::6816:3cd2, 2606:4700:10::6816:3dd2, 2606:4700:10::ac43:d98
Response IP 104.22.61.210
Found Yes
Hash d9dd6d8edfe6d8cce76f98ff24a506d656d5d88a529676e6baeb0e9a36924686
SimHash 31d62952ccec

Groups

*

Rule Path
Disallow /w/Property%3A
Disallow /w/Property%3A
Disallow /w/property%3A
Disallow /*?title=Property%3A
Disallow /*?title=Property%3A
Disallow /*?*&title=Property%3A
Disallow /*?*&title=Property%3A
Disallow /w/Special%3A
Disallow /w/Special%3A
Disallow /w/special%3A
Disallow /*?title=Special%3A
Disallow /*?title=Special%3A
Disallow /*?*&title=Special%3A
Disallow /*?*&title=Special%3A
Disallow /api.php
Disallow /cors/
Disallow /geoip$
Disallow /rest_v1/
Disallow /tags/
Disallow /*?action=
Disallow /*?*&action=
Disallow /*?feed=
Disallow /*?*&feed=
Disallow /*?from=
Disallow /*?*&from=
Disallow /*?mobileaction=
Disallow /*?*&mobileaction=
Disallow /*?oldid=
Disallow /*?*&oldid=
Disallow /*?printable=
Disallow /*?*&printable=
Disallow /*?redirect=
Disallow /*?*&redirect=
Disallow /*?uselang=
Disallow /*?*&uselang=
Disallow /*?useskin=
Disallow /*?*&useskin=
Disallow /*?variant=
Disallow /*?*&variant=
Disallow /*?veaction=
Disallow /*?*&veaction=
Disallow /*?filefrom=
Disallow /*?*&filefrom=
Disallow /*?fileuntil=
Disallow /*?*&fileuntil=
Disallow /*?navbox=
Disallow /*?*&navbox=
Disallow /*?pageuntil=
Disallow /*?*&pageuntil=
Disallow /*?pagefrom=
Disallow /*?*&pagefrom=
Disallow /*?diff=
Disallow /*?*&diff=
Disallow /*?curid=
Disallow /*?*&curid=
Disallow /*?search=
Disallow /*?*&search=
Disallow /*?section=
Disallow /*?*&section=

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

seekport

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

senutobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://runescape.wiki/images/sitemaps/index.xml

Comments

  • If you are using content from our site, please ensure that you are following the terms of our license (usually a Creative Commons license).
  • For more information, see https://meta.weirdgloop.org/w/Meta:Copyrights
  • SEO bots
  • Huawei's web crawler. Ignores Disallow and gets caught in loops accessing special pages. Produces the majority of uncachable requests.
  • AI bots