Log In Sign Up

charlesdoll.com.tw
robots.txt

Robots Exclusion Standard data for charlesdoll.com.tw

Resource Scan

Scan Details

Site Domain charlesdoll.com.tw
Base Domain charlesdoll.com.tw
Scan Status Ok
Last Scan2024-09-15T11:38:54+00:00
Next Scan 2024-10-15T11:38:54+00:00

Last Scan

Scanned2024-09-15T11:38:54+00:00
URL https://charlesdoll.com.tw/robots.txt
Domain IPs 45.76.204.136
Response IP 45.76.204.136
Found Yes
Hash c9fcc57ff454dccdee4ccf3a8a8cf86664d0f407a4f08fa1dffe87acf6f775e3
SimHash 031e4a00c51b

Groups

swebot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

bender

Rule Path
Disallow /

discobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

searchwebengine.net

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sindicebot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

findfiles.net

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

goodzer

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

fast enterprise crawler 6

Rule Path
Disallow /

sensis.com.au web crawler

Rule Path
Disallow /

worio bot heritrix

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

shoppimonagent

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120
Back to top

Other Records

Field Value
sitemap https://charlesdoll.com.tw/sitemap.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • http://www.trendiction.de/de/publisher/bot
  • https://serpstatbot.com
  • Disallow: SEOkicks-Robot
  • ---------------------------
  • END YOAST BLOCK
Back to top