tejikarao.com
robots.txt

Robots Exclusion Standard data for tejikarao.com

Resource Scan

Scan Details

Site Domain tejikarao.com
Base Domain tejikarao.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-25T04:50:11+00:00
Next Scan 2024-08-24T04:50:11+00:00

Last Successful Scan

Scanned2024-04-27T04:48:52+00:00
URL https://tejikarao.com/robots.txt
Redirect https://kodaisi.net/robots.txt
Redirect Domain kodaisi.net
Redirect Base kodaisi.net
Domain IPs 183.181.91.86
Redirect IPs 183.181.91.86
Response IP 183.181.91.86
Found Yes
Hash 1d82a04f87b377916da1ecc6d5b09108b978865da2f1b4ff6aabe72839a451bf
SimHash 72a77443b57a

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/themes/
Allow /*/*.js
Allow /*/*.css
Allow */rss2
Allow /post/
Allow /post/*
Allow /page/
Allow /page/*
Allow /tag/
Allow /tag/*
Allow /%tag%/*
Allow /%tag%
Allow /category/
Allow /category/*
Allow /%post%
Allow /%post%/
Allow /page/
Allow /%page%/
Allow /%page%
Allow /wp-content/themes/
Allow /*/*.js
Allow /*/*.css
Disallow /uploads/
Disallow /dcf*
Disallow /dcf*-jpg
Disallow /dcf*-*
Disallow /dsc_*-*
Disallow /dsc_*
Disallow /cgi-bin
Disallow /?s=
Disallow /*%26amp%3Bs%3D
Disallow /search
Disallow /author/
Disallow /?attachment_id=*
Disallow /*?attachment_id=*
Disallow /attachment-*
Disallow /attachment-*$
Disallow /wp-content/uploads/
Disallow /wp-*.png
Disallow /wp-*.jpg
Disallow /wp-*.jpeg
Disallow /wp-*.gif
Disallow /wp-*.svg
Disallow /wp-*.pdf
Disallow /dcf*
Disallow /dcf*-jpg
Disallow /*.png
Disallow /*.jpg
Disallow /*.jpeg
Disallow /*.gif
Disallow /*.svg
Disallow /*.pdf
Disallow /private/
Disallow /private*/
Disallow /cgi-bin

lurkmore

Rule Path
Disallow /

lurkmore

Rule Path
Disallow /

lurkmorebot

Rule Path
Disallow /

lurkmore-bot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

safedns search bot/nutch-1.9

Rule Path
Disallow /

kgbody/2.0

Rule Path
Disallow /

pinterest/0.1

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegel

Rule Path
Disallow /

applebot/

Rule Path
Disallow /

obot/

Rule Path
Disallow /

archive.is

Rule Path
Disallow /

archive.is

Rule Path
Disallow /

archive.de

Rule Path
Disallow /

archive.today

Rule Path
Disallow /

wget

Rule Path
Disallow /

wget

Rule Path
Disallow /

wget

Rule Path
Disallow /

sudo

Rule Path
Disallow /

curl

Rule Path
Disallow /

iria/1.07a

Rule Path
Disallow /

naofavicon4ie/1.*

Rule Path
Disallow /

webox/0.99

Rule Path
Disallow /

website explorer/0.9.*

Rule Path
Disallow /

wget/1.*.*

Rule Path
Disallow /

pockey-gethtml

Rule Path
Disallow /

internetlinkagent/3.1

Rule Path
Disallow /

wwwc/1.04

Rule Path
Disallow /

homepageclone

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow

ninja

Rule Path
Disallow /

riddler (http://riddler.io/about)

Rule Path
Disallow /

wgetrc

Rule Path
Disallow /

berry

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baidumobaider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

yeti

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

megalodon

Rule Path
Disallow /

yeti

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

advbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

publiclibraryarchive.org

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

abonti

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

mixbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

bubing

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bpimagewalker/2.0

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

feedbooster

Rule Path
Disallow /

nutch

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

exb language crawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bard

Rule Path
Disallow /

bingbot-chat/2.0

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgili bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kodaisi.net/wp-sitemap.xml
sitemap https://kodaisi.net/wp-sitemap-posts-post-1.xml
sitemap https://kodaisi.net/wp-sitemap-posts-page-1.xml
sitemap https://kodaisi.net/wp-sitemap-taxonomies-category-1.xml
sitemap https://kodaisi.net/wp-sitemap-taxonomies-post_tag-1.xml
sitemap https://kodaisi.net/sitemap-news.xml

Comments

  • http://kids.goo.ne.jp/tool/kgbody.php
  • http://pinterest.com/
  • aspiegel.com
  • aspiegel*
  • http://filterdb.iss.net/crawler/
  • http://archive.is/
  • http://archive.is/
  • http://archive.is/
  • http://archive.is/
  • YAHOO
  • NINJA bot
  • Amazon
  • USA Alexa: alexa.com/
  • China Baidu: www.baidu.com, www.baidu.jp
  • China Baidu: www.baidu.jp
  • China Baidu: www.baidu.jp
  • China Baidu: www.baidu.com, www.baidu.jp
  • China Yodao: www.yodao.com
  • Korea Naver: www.naver.com
  • Korea Naver: www.naver.com
  • Internet Archive
  • Korea Naver: www.naver.com
  • Korea Naver: www.naver.com
  • USA Alexa: alexa.com/
  • 2015.06.27 crawler for SentiOne
  • 2015.04.06 SEO indexer
  • 2015.02.10 AdvBot "classify web content"
  • 2015.01.30 XoviBot SEO bot
  • 2015.02.19 ??? parked domain
  • 2014.12.26. Internet Memory Research
  • 2014.09.26. SimilarTech, Lead Generation, Competitive Intelligence based on Web Tech Analysis
  • 2014.09.26. XOVI Suite, SEO & Online Marketing Tool
  • 2014.09.18. WebSearch
  • 2014.09.11. The web search API
  • SEO services
  • panscient.com
  • tiscali.it search bot
  • search engine
  • search engine
  • Mixdata : data for big business
  • chinese search engine
  • chinese search engine
  • scalable, fully distributed crawler
  • ??? search engine
  • search engine
  • the Internet Archive's open-source, extensible, scalable, archival-quality Web crawler
  • kostenlose Backlinkchecker von Torsten Rückert Internetdiestleistungen
  • part of Ware Bay Best Buys Search engine
  • Web crawler
  • analyses the structure of the WWW
  • search engine
  • seo
  • brand protection
  • seo
  • seo
  • search engine
  • seo
  • plagiarism check
  • search engine www.sengine.info
  • news
  • Apache Nutch based
  • news portal
  • seo moz
  • seo
  • seo
  • language
  • BEGIN - Added by ChatBot Blocker by CellarWeb plugin
  • Blocks ChatGPT bot scanning
  • Blocks Bard bot scanning
  • Blocks Bing bot scanning
  • Blocks Common Crawl bot scanning
  • Blocks omgili bot scanning
  • Blocks omgilibot bot scanning
  • END - Added by ChatBot Blocker by CellarWeb plugin
  • Block archive.org bots