miixtapechiick.com
robots.txt

Robots Exclusion Standard data for miixtapechiick.com

Resource Scan

Scan Details

Site Domain miixtapechiick.com
Base Domain miixtapechiick.com
Scan Status Ok
Last Scan2024-11-18T00:47:17+00:00
Next Scan 2024-11-25T00:47:17+00:00

Last Scan

Scanned2024-11-18T00:47:17+00:00
URL https://miixtapechiick.com/robots.txt
Domain IPs 166.62.111.210
Response IP 166.62.111.210
Found Yes
Hash 8d49f29e8e1c567f4ad276a784312b8501a246cedd9baad201101b383bf0f02f
SimHash e53a4965e891

Groups

grapeshot

Rule Path
Disallow /sitemap/

*

Rule Path
Disallow /wp-admin
Disallow /wp-content/plugins/
Disallow /core/
Disallow /dex/
Disallow /cgi-bin/
Disallow /includes/
Disallow /flash/
Disallow /plugins/
Disallow /system/
Disallow /widgets/
Disallow /assets/
Disallow /music/wp-content/
Disallow /music/
Disallow /api/
Disallow /static/js/
Disallow /static/css/
Disallow /js/
Disallow /css/
Disallow /tv/
Disallow /?s=
Disallow /page/
Disallow /tag/
Disallow /search/
Disallow /wp-content/uploads/
Disallow /category/

Other Records

Field Value
crawl-delay 20

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claude-web-embed

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

Other Records

Field Value
sitemap https://miixtapechiick.com/sitemap.xml
sitemap https://miixtapechiick.com/news-sitemap.xml
sitemap https://miixtapechiick.com/sitemap.xml
sitemap https://miixtapechiick.com/news-sitemap.xml