kurnio.com
robots.txt

Robots Exclusion Standard data for kurnio.com

Resource Scan

Scan Details

Site Domain kurnio.com
Base Domain kurnio.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-03T10:06:45+00:00
Next Scan 2024-12-02T10:06:45+00:00

Last Successful Scan

Scanned2024-08-05T10:05:05+00:00
URL https://kurnio.com/robots.txt
Domain IPs 172.104.186.57, 2400:8901::f03c:93ff:feca:97b3
Response IP 172.104.186.57
Found Yes
Hash 579d4a13949a6ba42d695311d6b03f450a4e6eb190c111a268ec126283c32153
SimHash 495c5740e7bb

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /cgi-bin/
Disallow /tag/
Disallow /?s=*
Allow /wp-admin/admin-ajax.php

applebot
garlikcrawler
mappy
cliqzbot
ltx71 - (http://ltx71.com/)
daum
grapeshot

Rule Path
Disallow /

special_archiver
proximic
blexbot
cula
velenpublicwebcrawler

Rule Path
Disallow /

semrushbot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
splitsignalbot
semrushbot-coub
ahrefsbot
ahrefssiteaudit
mj12bot
rogerbot
spbot
ia_archiver
archive.org_bot
extlinksbot
linkdexbot
serpstatbot
dotbot
megaindex.ru
exabot
gigabot
sitebot

Rule Path
Disallow /

mail.ru_bot
mauibot
linguee bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.kurnio.com/sitemap.xml
sitemap https://www.kurnio.com/post-sitemap.xml

Comments

  • Block Marketing Tools Bots
  • Block Web Stats Bots
  • Block SEO Tools Bots
  • Block Annoying Bots