Log In Sign Up

kiamedia.com
robots.txt

Robots Exclusion Standard data for kiamedia.com

Resource Scan

Scan Details

Site Domain kiamedia.com
Base Domain kiamedia.com
Scan Status Ok
Last Scan2024-09-29T08:35:19+00:00
Next Scan 2024-10-29T08:35:19+00:00

Last Scan

Scanned2024-09-29T08:35:19+00:00
URL https://kiamedia.com/robots.txt
Domain IPs 3.98.252.51, 35.183.247.62
Response IP 35.183.247.62
Found Yes
Hash 2e112e812ece980af477b696461aded8889fffd3f17375aadbb9c7bf4f6415c9
SimHash f0771949c330

Groups

ahrefsbot
ezooms
sistrix
mj12bot
megaindex.ru
megaindex.com
petalbot

Rule Path
Disallow /

ccbot
claudebot
claude-web
chatgpt-user
gptbot
google-extended
applebot-extended
anthropic-ai
omgilibot
omgili
facebookbot
diffbot
bytespider
imagesiftbot
perplexitybot
cohere-ai

Rule Path
Disallow /

*

Rule Path
Disallow /*/*/print/
Disallow /*/*/download/*
Disallow /content/jqueryui/
Disallow /Content/jqueryui/
Disallow /*/*/basket/*
Disallow /content/images/icons/
Disallow /*/*/basket/
Disallow /*/*/newsalert/
Disallow /*/*/search/
Disallow /*/*/presskits/

Other Records

Field Value
crawl-delay 2
Back to top

Comments

  • AI Data Scrapers
  • ----------------
  • This list of bots based on https://darkvisitors.com/ and https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
  • Info on the different bots is possible at https://darkvisitors.com/
Back to top