Log In Sign Up

cbshow.org.tw
robots.txt

Robots Exclusion Standard data for cbshow.org.tw

Resource Scan

Scan Details

Site Domain cbshow.org.tw
Base Domain cbshow.org.tw
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-12-12T09:04:48+00:00
Next Scan 2025-12-19T09:04:48+00:00

Last Successful Scan

Scanned2025-11-09T18:52:02+00:00
URL https://www.cbshow.org.tw/robots.txt
Domain IPs 60.249.97.235
Response IP 60.249.97.235
Found Yes
Hash 92fe774e9983d4e90a7ee54c45af4aff71d0717f877c4731628be205efc10890
SimHash 9800a943e962

Groups

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-ocob

Rule Path
Disallow /

semrushbot-ft

Rule Path
Disallow /

rytebot

Rule Path
Disallow /

semrushbot-esi

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

newspaper/0.2.8

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/7.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; semrushbot/7~bl; +http://www.semrush.com/bot.html)

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

mozilla/5.0 (compatible; dubbotbot/0.2; +http://dubbot.com)

Rule Path
Disallow /

applebot

Rule Path
Disallow /

mozilla/5.0 (compatible; mj12bot/v1.4.8; http://mj12bot.com/)

Rule Path
Disallow /

blp_bbot/0.1

Rule Path
Disallow /

mozilla/5.0 (compatible; linux x86_64; mail.ru_bot/2.0; +http://go.mail.ru/help/robots)

Rule Path
Disallow /

sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

eyemonit_bot_version_0.1_(http://www.eyemon.it/)

Rule Path
Disallow /

mozilla/5.0 (compatible; blexbot/1.0; +http://webmeup-crawler.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; barkrowler/0.9; +https://babbar.tech/crawler)

Rule Path
Disallow /

mozilla/5.0 (compatible; dataforseobot/1.0; +https://dataforseo.com/dataforseo-bot)

Rule Path
Disallow /

mozilla/5.0 (compatible; adsbot/3.1; +https://seostar.co/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/bots/)

Rule Path
Disallow /

mozilla/5.0 (compatible;petalbot;+https://webmaster.petalsearch.com/site/petalbot)

Rule Path
Disallow /

mozilla/5.0 (compatible; neevabot/1.0; +https://neeva.com/neevabot)

Rule Path
Disallow /

mozilla/5.0 (compatible; mojeekbot/0.10; +https://www.mojeek.com/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; coccocbot-image/1.0; +http://help.coccoc.com/searchengine)

Rule Path
Disallow /

zoombot (linkbot 1.0 http://suite.seozoom.it/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; megaindex.ru/2.0; +http://megaindex.com/crawler)

Rule Path
Disallow /

pyspider/0.3.10 (+http://pyspider.org/)

Rule Path
Disallow /

mozilla/5.0 (compatible; dotbot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)

Rule Path
Disallow /

academicbotrtu (https://academicbot.rtu.lv; mailto:caps@rtu.lv)

Rule Path
Disallow /

mauibot (crawler.feedback+gamma@gmail.com)

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

inetdex-bot/1.5 (mozilla/5.0; https://inetdex.com/; info at inetdex dot com)

Rule Path
Disallow /

screaming frog seo spider/16.6

Rule Path
Disallow /

mozilla/5.0 (compatible; seznambot/3.2-test1; +http://napoveda.seznam.cz/en/seznambot-intro/)

Rule Path
Disallow /

mozilla/5.0 (compatible; linespider/1.1; +https://lin.ee/4dwxkth)

Rule Path
Disallow /

zoominfobot (zoominfobot at zoominfo dot com)

Rule Path
Disallow /

mozilla/5.0 (compatible; qwantify/2.4w; +https://www.qwant.com/)

Rule Path
Disallow /

screaming frog seo spider/16.7

Rule Path
Disallow /

mozilla/5.0 (compatible; mojeekbot/0.11; +https://www.mojeek.com/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; seznambot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)

Rule Path
Disallow /

awariosmartbot/1.0 (+https://awario.com/bots.html; bots@awario.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; siteauditbot/0.97; +http://www.semrush.com/bot.html)

Rule Path
Disallow /

legistorm bot (http://www.legistorm.com/legibot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; infotigerbot/1.9; +https://infotiger.com/bot)

Rule Path
Disallow /

mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

ccbot/2.0 (https://commoncrawl.org/faq/)

Rule Path
Disallow /

buck/2.3.2; (+https://app.hypefactors.com/media-monitoring/about.html)

Rule Path
Disallow /

turnitin (https://bit.ly/2uvnfoq)

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

awariosmartbot/1.0 (+https://awario.com/bots.html; bots@awario.com)

Rule Path
Disallow /

awariorssbot/1.0 (+https://awario.com/bots.html; bots@awario.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; obot/2.3.1; http://www.xforce-security.com/crawler/)

Rule Path
Disallow /

mozilla/5.0 (compatible) semanticscholarbot (+https://www.semanticscholar.org/crawler)

Rule Path
Disallow /

mozilla/5.0 (compatible; coccocbot-web/1.0; +http://help.coccoc.com/searchengine)

Rule Path
Disallow /

expanse, a palo alto networks company, searches across the global ipv4 space multiple times per day to identify customers&

Product Comment
expanse, a palo alto networks company, searches across the global ipv4 space multiple times per day to identify customers& 39; presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com
Rule Path
Disallow /

screaming frog seo spider/17.2

Rule Path
Disallow /

linabot

Rule Path
Disallow /

scrapy/1.7.4 (+https://scrapy.org)

Rule Path
Disallow /

python-requests/2.28.2

Rule Path
Disallow /

mozilla/5.0 (linux; android 5.0) applewebkit/537.36 (khtml, like gecko) mobile safari/537.36 (compatible; bytespider; https://zhanzhang.toutiao.com/)

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) gecko/20170101 firefox/67.0

Rule Path
Disallow /

mozilla/5.0 (linux; android 5.0) applewebkit/537.36 (khtml, like gecko) mobile safari/537.36 (compatible; bytespider; https://zhanzhang.toutiao.com/)

Rule Path
Disallow /

mozilla/5.0 (linux; android 5.0) applewebkit/537.36 (khtml, like gecko) mobile safari/537.36 (compatible; bytespider; spider-feedback@bytedance.com)

Rule Path
Disallow /

screaming frog seo spider/18.4

Rule Path
Disallow /

screaming frog seo spider/19.0

Rule Path
Disallow /

mozilla/5.0 (compatible; semrushbot; +http://www.semrush.com/bot.html)

Rule Path
Disallow /

mozilla/5.0 (compatible; dataprovider.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; seznambot/4.0; +http://napoveda.seznam.cz/seznambot-intro/)

Rule Path
Disallow /

mozilla/5.0 (compatible; dataforseobot/1.0; +https://dataforseo.com/dataforseo-bot)

Rule Path
Disallow /

gulper web bot 0.2.4 (www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/link/gulperbot)

Rule Path
Disallow /

mozilla/5.0 (compatible; paqlebot/2.0; +http://www.paqle.dk/about/paqlebot)

Rule Path
Disallow /

mediatoolkitbot (complaints@mediatoolkit.com)

Rule Path
Disallow /

mozilla/5.0 (macintosh; intel mac os x 10_10_1) applewebkit/600.2.5 (khtml, like gecko) version/8.0.2 safari/600.2.5 (amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

mozilla/5.0 (compatible; seekportbot; +https://bot.seekport.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; yeti/1.1; +https://naver.me/spd)

Rule Path
Disallow /

mozilla/5.0 applewebkit/537.36 (khtml, like gecko; compatible; claudebot/1.0; +claudebot@anthropic.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; openindexspider; +https://www.openindex.io/saas/about-our-spider/)

Rule Path
Disallow /

domcopbot (https://www.domcop.com/bot)

Rule Path
Disallow /

mozilla/5.0 (compatible; awariobot/1.0; +https://awario.com/bots.html)

Rule Path
Disallow /

mozilla/5.0 applewebkit/537.36 (khtml, like gecko; compatible; gptbot/1.0; +https://openai.com/gptbot)

Rule Path
Disallow /

mozilla/5.0 applewebkit/537.36 (khtml, like gecko; compatible; claudebot/1.0; +claudebot@anthropic.com)

Rule Path
Disallow /

*

Rule Path
Disallow */*login*
Disallow */*Login*
Disallow */intranet/*
Disallow */include*
Disallow /*upload*
Disallow /*upfile*
Disallow */*webservice*
Disallow */api/*
Disallow /tis
Disallow /tis2005
Disallow /meetdoc
Disallow /internet/eip*
Disallow /internet/e-vip
Disallow /ePassport
Disallow /dvrmonitor
Disallow /isocenter
Disallow /LearningPassport
Disallow /CloverMonthly
Disallow */Login.aspx
Disallow */*.doc$
Disallow */*.docx$
Disallow */*.pdf$
Disallow */*.ppt$
Disallow */*.pptx$
Disallow */*.xls$
Disallow */*.xlsx$
Disallow */*.xml$
Disallow */*.ini$
Disallow */*.txt$
Disallow */*.zip$
Disallow */*.7z$
Disallow */*.rar$
Disallow */*.gif$
Disallow */*.asp$
Allow /changbin
Allow /sitemap.xml$
Back to top

Comments

  • To block SemrushBot from crawling your site for different SEO and technical issues:
  • To block SemrushBot from crawling your site for Backlink Audit tool:
  • To block SemrushBot from crawling your site for On Page SEO Checker tool and similar tools:
  • To block SemrushBot from checking URLs on your site for SWA tool:
  • To block SplitSignalBot from crawling your site for SplitSignal tool:
  • To block SemrushBot-OCOB from crawling your site for Content Toolkit:
  • To block SemrushBot-FT from crawling your site for Plagiarism Checker and similar tools:
  • To block RyteBot from crawling your site for tools provided by Ryte.com:
  • To block SemrushBot-ESI from crawling your site for services provided by Semrush Enterprise Site Intelligence:
  • Disallow: */*.jpg$
  • Disallow: */*.jpeg$
  • Disallow: */*.png$
Back to top