kantotin.net
robots.txt

Robots Exclusion Standard data for kantotin.net

Resource Scan

Scan Details

Site Domain kantotin.net
Base Domain kantotin.net
Scan Status Ok
Last Scan2025-03-28T20:38:46+00:00
Next Scan 2025-04-27T20:38:46+00:00

Last Scan

Scanned2025-03-28T20:38:46+00:00
URL https://kantotin.net/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.80.1
Found Yes
Hash 129fe0c44abcb678b7dd49d72d1895b7a18d8e241369707caf59444c4b02ad85
SimHash f63473c0c4d2

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Allow /*/*.css
Allow /*/*.js
Allow /wp-content/uploads/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /readme.html
Disallow /license.txt
Disallow /xmlrpc.php
Disallow /wp-login.php
Disallow /wp-register.php
Disallow *?attachment_id=
Disallow /wp-content/plugins/
Disallow /refer/

*

Rule Path
Disallow /wp-json/
Disallow /?rest_route=

*

Rule Path Comment
Disallow /search/ -
Disallow /?s= -
Disallow /?filter= -
Disallow /?s=%7Bsearch_term_string%7D -
Disallow /?filter=longest -
Disallow /?filter=longest !
Disallow /?filter=random -
Disallow /?filter=latest -
Disallow /?filter=popular -
Disallow /?filter=most-viewed !
Disallow /?filter=most-viewed -
Disallow /?cid= -
Disallow /?eid= -
Disallow /?filter=random !
Disallow /?mode=myaccount -
Disallow /cdn-cgi/challenge-platform/h/b -
Disallow /cdn-cgi/challenge-platform/h/b/scripts/alpha/invisible.js?ts= -
Disallow /wp-login.php?action=lostpassword -
Disallow /search/%7Bsearch_term_string%7D/feed/rss2/ -
Disallow /cdn-cgi/ -
Disallow /imdb/?action=sign-in -
Disallow /imdb/?action=log-in -
Disallow /?get= -
Disallow / !
Disallow /?action=sign-in -
Disallow /cdn-cgi/challenge-platform/h/b/scripts/cb/invisible.js?cb=7b2cc8913c5b0384 -
Disallow /cdn-cgi/challenge-platform/h/g -
Disallow /cdn-cgi/challenge-platform/h/b/scripts/cb/invisible.js?cb=799acad74e5d6fd1 -
Disallow /cdn-cgi/challenge-platform/ -
Disallow /cdn-cgi/challenge-platform/h/b/ -
Disallow /cdn-cgi/ -
Disallow /cdn-cgi/challenge-platform/h/b/scripts/cb/invisible.js?cb=770786252a01e26f -
Disallow /hello-world/ -
Disallow /9bcpwf/land-pride-380-166a.html -
Disallow /wp-content/plugins/webpushr-web-push-notifications/sdk_files/webpushr-sw.js.php -
Disallow /9bcpwf/principles-of-hematology-pdf.html -
Disallow /cdn-cgi/challenge-platform/h/g/scripts/cb/invisible.js?cb=7b7a2c3a38634791 -
Disallow /cdn-cgi/challenge-platform/h/g/scripts/cb/invisible.js?cb=7b84445ba88a13eb -
Disallow /9bcpwf/journal-bearing-calculation-pdf.html -
Disallow /wp-content/plugins/perfecty-push-notifications/public/js -
Disallow /9bcpwf/journal-bearing-calculation-pdf.html -
Disallow /9bcpwf/free-printable-resistance-band-workout-chart.html -
Disallow /cdn-cgi/challenge-platform/h/b/scripts/cb/invisible.js?cb=7b58c556f81907b8 -
Disallow /a84omvy/dynex-tv-says-no-signal.html -
Disallow /9bcpwf/new-holland-16x16-transmission-problems.html -
Disallow /a84omvy/scenes-for-teenage-actors-pdf.html -
Disallow /a84omvy/omnirom.html -
Disallow /a84omvy/ -
Allow /*?$ -
Disallow /*? -

*

Rule Path
Disallow *?s=*
Disallow *?p=*
Disallow *%26p%3D*
Disallow *%26preview%3D*

*

Rule Path
Disallow /feed/
Disallow /feed/$
Disallow /comments/feed
Disallow */feed
Disallow */feed$
Disallow /?feed=
Disallow /wp-feed

*

Rule Path
Disallow /trackback/
Disallow */comments$
Disallow */trackback
Disallow */trackback$
Disallow /wp-comments
Disallow /wp-trackback
Disallow */replytocom%3D

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /cart/

*

Rule Path
Disallow /checkout/

*

Rule Path
Disallow /my-account/

*

Rule Path
Disallow /login/

*

Rule Path
Disallow /*?orderby=price
Disallow /*?orderby=rating
Disallow /*?orderby=date
Disallow /*?orderby=price-desc
Disallow /*?orderby=popularity
Disallow /*?filter
Disallow /*?orderby=title
Disallow /*?orderby=desc
Disallow /*add-to-cart%3D*
Disallow /*add_to_wishlist%3D*
Disallow /*?paged=&count=*
Disallow /*?count=*

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /wp-content/uploads/

applebot

Rule Path
Allow /

yandex

Rule Path
Allow /

yandeximages

Rule Path
Allow /wp-content/uploads/

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

qwantify

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider/2.0

Rule Path
Allow /

baiduspider-video

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

sogou spider

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

sosospider

Rule Path
Allow /

sosospider+

Rule Path
Allow /

sosospider/2.0

Rule Path
Allow /

yodao

Rule Path
Allow /

youdao

Rule Path
Allow /

youdaobot

Rule Path
Allow /

youdaobot/1.0

Rule Path
Allow /

naverbot

Rule Path
Allow /

seznambot

Rule Path
Allow /

facebook

Rule Path
Allow /

facebookplatform/1.0

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

facebookexternalhit/1.0

Rule Path
Allow /

facebookexternalhit/1.1

Rule Path
Allow /

facebookscraper

Rule Path
Allow /

facebot/1.0

Rule Path
Allow /

visionutils/0.2

Rule Path
Allow /

datagnionbot/1.0

Rule Path
Allow /

instagrambot

Rule Path
Allow /

whatsapp bot

Rule Path
Allow /

telegrambot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

linkedinbot/1.0

Rule Path
Allow /

pinterest bot

Rule Path
Allow /

pinterest/0.1

Rule Path
Allow /

pinterest/0.2

Rule Path
Allow /

discordbot

Rule Path
Allow /

*

Rule Path
Disallow /*.webp$

*

Rule Path
Disallow /*.jpg$

*

Rule Path
Disallow /*.png$

*

Rule Path
Disallow /*.gif$

*

Rule Path
Disallow /*.pdf$

*

Rule Path
Disallow /*.docx$

*

Rule Path
Disallow /*.html$

*

Rule Path
Disallow /*.php$

dotbot

Rule Path
Disallow /

giftghostbot

Rule Path
Disallow /

seznam

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

dataprovider/6.101

Rule Path
Disallow /

dataprovidersiteexplorer

Rule Path
Disallow /

dazoobot/1.0

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

domainstatsbot/1.0

Rule Path
Disallow /

dotbot/1.1

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

ecommercebot

Rule Path
Disallow /

expertsearchspider

Rule Path
Disallow /

feedbin

Rule Path
Disallow /

fetch/2.0a

Rule Path
Disallow /

ffbot/1.0

Rule Path
Disallow /

focusbot/1.1

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

huaweisymantecspider/1.0

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

lipperheylinkexplorer

Rule Path
Disallow /

lssrocketcrawler/1.0

Rule Path
Disallow /

lyt.srv1.5

Rule Path
Disallow /

miadev/0.0.1

Rule Path
Disallow /

najdi.si/3.1

Rule Path
Disallow /

bountiibot

Rule Path
Disallow /

experibot_v1

Rule Path
Disallow /

bixocrawler

Rule Path
Disallow /

bixocrawler testcrawler

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

crowsnest/0.5

Rule Path
Disallow /

cukbot

Rule Path
Disallow /

dataprovider/6.92

Rule Path
Disallow /

dblbot/1.0

Rule Path
Disallow /

diffbot/0.1

Rule Path
Disallow /

digg deeper/v1

Rule Path
Disallow /

discobot/1.0

Rule Path
Disallow /

discobot/1.1

Rule Path
Disallow /

discobot/2.0

Rule Path
Disallow /

discoverybot/2.0

Rule Path
Disallow /

dlvr.it/1.0

Rule Path
Disallow /

domainstatsbot/1.0

Rule Path
Disallow /

drupact/0.7

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

fastbot crawler beta 2.0

Rule Path
Disallow /

fastbot crawler beta 4.0

Rule Path
Disallow /

feedly social

Rule Path
Disallow /

feedly/1.0

Rule Path
Disallow /

feedlybot/1.0

Rule Path
Disallow /

feedspot

Rule Path
Disallow /

feedspotbot/1.0

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

classbot

Rule Path
Disallow /

cispa vulnerability notification

Rule Path
Disallow /

cirrusexplorer/1.1

Rule Path
Disallow /

checksem/nutch-1.10

Rule Path
Disallow /

catchbot/5.0

Rule Path
Disallow /

catchbot/3.0

Rule Path
Disallow /

catchbot/2.0

Rule Path
Disallow /

catchbot/1.0

Rule Path
Disallow /

camontspider/1.0

Rule Path
Disallow /

buzzbot/1.0

Rule Path
Disallow /

buzzbot

Rule Path
Disallow /

businessseek.biz_spider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

fyberspider/1.3

Rule Path
Disallow /

findlinks/1.1.6-beta5

Rule Path
Disallow /

g2reader-bot/1.0

Rule Path
Disallow /

findlinks/1.1.6-beta6

Rule Path
Disallow /

findlinks/2.0

Rule Path
Disallow /

findlinks/2.0.1

Rule Path
Disallow /

findlinks/2.0.2

Rule Path
Disallow /

findlinks/2.0.4

Rule Path
Disallow /

findlinks/2.0.5

Rule Path
Disallow /

findlinks/2.0.9

Rule Path
Disallow /

findlinks/2.1

Rule Path
Disallow /

findlinks/2.1.5

Rule Path
Disallow /

findlinks/2.1.3

Rule Path
Disallow /

findlinks/2.2

Rule Path
Disallow /

findlinks/2.5

Rule Path
Disallow /

findlinks/2.6

Rule Path
Disallow /

ffbot/1.0

Rule Path
Disallow /

findlinks/1.0

Rule Path
Disallow /

findlinks/1.1.3-beta8

Rule Path
Disallow /

findlinks/1.1.3-beta9

Rule Path
Disallow /

findlinks/1.1.4-beta7

Rule Path
Disallow /

findlinks/1.1.6-beta1

Rule Path
Disallow /

findlinks/1.1.6-beta1 yacy

Rule Path
Disallow /

findlinks/1.1.6-beta2

Rule Path
Disallow /

findlinks/1.1.6-beta3

Rule Path
Disallow /

findlinks/1.1.6-beta4

Rule Path
Disallow /

bixo

Rule Path
Disallow /

bixolabs/1.0

Rule Path
Disallow /

crawlera/1.10.2

Rule Path
Disallow /

dataprovider site explorer

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kantotin.net/post-sitemap.xml
sitemap https://kantotin.net/sitemap_index.xml
sitemap https://kantotin.net/sitemap.xml

Comments

  • Advanced Wordpress
  • Prevent Crawling of WordPress JSON API Endpoints
  • Block Search URLs /search/ and /?s=
  • Block Parameters
  • Block Feed
  • Block Spam Directories
  • Block archive.org bots
  • Block Chatgpt
  • Block Cart Page
  • Block Checkout Page
  • Block My Account Page
  • Block Login Page
  • Block Woocommerce Parameters
  • Yoast Sitemap Link
  • XML Sitemaps Sitemap Link
  • Block Ahrefs Crawler
  • Block Semrush Crawler
  • Block Moz Crawler
  • Block Majestic Crawler
  • Block Xenu Crawler
  • Allow Google Bot
  • Allow Google Images Bot
  • Allow Google Media Partners Bot
  • Allow Google AdsBot Bot
  • Allow Google Mobile Bot
  • Allow Bing Bot
  • Allow MSN Bot
  • Allow MSNBot Media Bot
  • Allow Apple Bot
  • Allow Yandex Bot
  • Allow Yandex Images Bot
  • Allow Yahoo Search (Slurp bot)
  • Allow DuckDuckGo Bot
  • Allow Qwant Bot
  • Allow Baidu/Sogou/Soso/Youdao Bot
  • Allow Naver Bot
  • Allow Seznam Bot
  • Allow Facebook Bot
  • Allow Instagram Bot
  • Allow Whatsapp Bot
  • Allow Telegram Bot
  • Allow Twitter Bot
  • Allow Linkedin Bot
  • Allow Pinterest Bot
  • Allow Discord Bot
  • Block Webp Images
  • Block Jpg Images
  • Block Png Images
  • Block Gif Images
  • Block PDF Files
  • Block DOCX Files
  • Block Html Files
  • Block Php Files
  • Block Scrapper Bots

Warnings

  • 8 invalid lines.
  • `post-sitemap` is not a known field.