Log In Sign Up

mixes.wiki
robots.txt

Robots Exclusion Standard data for mixes.wiki

Resource Scan

Scan Details

Site Domain mixes.wiki
Base Domain mixes.wiki
Scan Status Ok
Last Scan2024-10-04T02:38:45+00:00
Next Scan 2024-10-18T02:38:45+00:00

Last Scan

Scanned2024-10-04T02:38:45+00:00
URL https://mixes.wiki/robots.txt
Domain IPs 104.21.77.13, 172.67.203.23, 2606:4700:3032::ac43:cb17, 2606:4700:3035::6815:4d0d
Response IP 104.21.77.13
Found Yes
Hash fa426353aba3e2b29fbac990235cd73aa7c7e93f180c7fe577db5da857247797
SimHash 7a4c61408ff3

Groups

*

Rule Path
Disallow /db/
Disallow /list-artist-content/
Disallow /tools/
Disallow action%3D
Disallow .mp3
Disallow MixesDB_Search.php
Disallow MixesDB_Search.xml
Disallow favicon.ico
Disallow info.php
Disallow ref.php
Disallow /w/Template
Disallow /w/Special
Disallow /w/Special%3AContributions
Disallow /w/Talk
Disallow /w/Category%3ADate
Disallow /w/Category%3AHidden_categories
Disallow /w/Category%3AAudio_Players
Disallow /w/Category%3ADirect_Players
Disallow /w/Category%3AInternet_Archive
Disallow /w/Category%3AMixcloud
Disallow /w/Category%3AOfficial.FM
Disallow /w/Category%3AOther_Video_Players
Disallow /w/Category%3APodOmatic
Disallow /w/Category%3APodbean.com
Disallow /w/Category%3ASkip_Main_Page
Disallow /w/Category%3ASoundCloud
Disallow /w/Category%3AVideo_Players
Disallow /w/Category%3AVimeo
Disallow /w/Category%3AYouTube
Disallow /w/Category%3Ab%40_TV
Disallow /w/Category%3Aplay.fm
Disallow /w/Category%3ACharts
Disallow /w/Category%3AApple_Music
Disallow /w/Category%3ASpotify_Podcasts
Disallow /w/Category%3ASpotify_Playlists
Disallow /w/Category%3ABeatport_Charts
Disallow /w/Category%3AMisc_Charts
Disallow /w/Category%3AJuno_Charts
Disallow /w/Category%3AResident_Advisor_Charts
Disallow /w/Category%3AResident_Advisor_Charts_%28Tops%29
Disallow /w/Mixes.wiki%3AExplorer
Disallow /w/Mixes.wiki%3ARegular_categories
Disallow /w/Mixes.wiki%3APlayer_search
Disallow /w/Category%3ADate
Disallow /w/Category%3AHidden_categories
Disallow /w/Category%3AAudio_Players
Disallow /w/Category%3ADirect_Players
Disallow /w/Category%3AInternet_Archive
Disallow /w/Category%3AMixcloud
Disallow /w/Category%3AOfficial.FM
Disallow /w/Category%3AOther_Video_Players
Disallow /w/Category%3APodOmatic
Disallow /w/Category%3APodbean.com
Disallow /w/Category%3ASkipMP
Disallow /w/Category%3ASoundCloud
Disallow /w/Category%3AVideo_Players
Disallow /w/Category%3AVimeo
Disallow /w/Category%3AYouTube
Disallow /w/Category%3Ab%40_TV
Disallow /w/Category%3Aplay.fm
Disallow /w/Category%3ACharts
Disallow /w/Category%3ABeatport_Charts
Disallow /w/Category%3AMisc_Charts
Disallow /w/Category%3AJuno_Charts
Disallow /w/Category%3AResident_Advisor_Charts
Disallow /w/Category%3AResident_Advisor_Charts_%28Tops%29
Disallow Mixes.wiki%3Explorer
Disallow Mixes.wiki%3ARegular_categories
Disallow Mixes.wiki%3Player_search
Disallow /w/index.php?title=Category%3ADate
Disallow /w/index.php?title=Category%3AHidden_categories
Disallow /w/index.php?title=Category%3AAudio_Players
Disallow /w/index.php?title=Category%3ADirect_Players
Disallow /w/index.php?title=Category%3AInternet_Archive
Disallow /w/index.php?title=Category%3AMixcloud
Disallow /w/index.php?title=Category%3AOfficial.FM
Disallow /w/index.php?title=Category%3AOther_Video_Players
Disallow /w/index.php?title=Category%3APodOmatic
Disallow /w/index.php?title=Category%3APodbean.com
Disallow /w/index.php?title=Category%3ASkip_Main_Page
Disallow /w/index.php?title=Category%3ASoundCloud
Disallow /w/index.php?title=Category%3AVideo_Players
Disallow /w/index.php?title=Category%3AVimeo
Disallow /w/index.php?title=Category%3AYouTube
Disallow /w/index.php?title=Category%3Ab%40_TV
Disallow /w/index.php?title=Category%3Aplay.fm
Disallow /w/index.php?title=Category%3ACharts
Disallow /w/index.php?title=Category%3AApple_Music
Disallow /w/index.php?title=Category%3ASpotify_Podcasts
Disallow /w/index.php?title=Category%3ASpotify_Playlists
Disallow /w/index.php?title=Category%3ABeatport_Charts
Disallow /w/index.php?title=Category%3AMisc_Charts
Disallow /w/index.php?title=Category%3AJuno_Charts
Disallow /w/index.php?title=Category%3AResident_Advisor_Charts
Disallow /w/index.php?title=Category%3AResident_Advisor_Charts_%28Tops%29
Disallow /w/index.php?title=Mixes.wiki%3AExplorer
Disallow /w/index.php?title=Mixes.wiki%3ARegular_categories
Disallow /w/index.php?title=Mixes.wiki%3APlayer_search
Disallow /*?*&action=history
Disallow /*?*&action=info
Disallow /*?*&action=edit
Disallow /*?*&action=submit
Disallow /*?*&action=purge
Disallow /*?*oldid=
Disallow sitemap.txt

Other Records

Field Value
crawl-delay 4

ia_archiver
ahrefsbot
aihitbot
alexa
alphabot
asks
blexbot
baiduspider-render
catexplorador
ccbot
careerbot
cipacrawler
doc
cliqzbot
dataxu
domaincheck.io
dotbot
dotmic
download ninja
fetch
grapeshot
grapeshotcrawler
httrack
lightspeedsystemscrawler
mj12bot
msiecrawler
mail.ru_bot
mauibot
megaindex
microsoft.url.control
netcraftsurveyagent
nimbostratus
npbot
offline explorer
orthogaffe
pinterest
plukkie
proximic
queryseekerspider
riddler
semrushbot
seokicks
seokicks-robot
sistrix
safednsbot
searchmetricsbot
seekport crawler
semrushbot
semrushbot-ba
sitesnagger
sogou
teleport
teleportpro
ubicrawler
vagabondo
wbsearchbot
webcopier
webreaper
webstripper
webzip
xenu
xovibot
xovibot
yandex
yandexbot
yisouspider
zao
zealbot
zoominfobot
zyborg
coccocbot
grub-client
icjobs
ips-agent
istellabot
k2spider
larbin
libwww
linkdexbot
linko
magpie-crawler
masscan
netestate ne crawler
nsrbot
obot
probethenet
proximic
rogerbot

Rule Path
Disallow /

wget

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.mixes.wiki/sitemap/sitemap-index-mixes.wiki.xml
Back to top

Comments

  • Sorry, wget in its recursive mode is a frequent problem.
  • Please read the man page and use it properly; there is a
  • --wait option you can use to set the delay between hits,
  • for instance.
Back to top

Warnings

  • 1 invalid line.
Back to top