Log In Sign Up

chartable.com
robots.txt

Robots Exclusion Standard data for chartable.com

Resource Scan

Scan Details

Site Domain chartable.com
Base Domain chartable.com
Scan Status Ok
Last Scan2024-09-20T00:46:46+00:00
Next Scan 2024-10-20T00:46:46+00:00

Last Scan

Scanned2024-09-20T00:46:46+00:00
URL https://chartable.com/robots.txt
Domain IPs 104.22.48.104, 104.22.49.104, 172.67.11.72, 2606:4700:10::6816:3068, 2606:4700:10::6816:3168, 2606:4700:10::ac43:b48
Response IP 172.67.11.72
Found Yes
Hash 4a9cec93b93b2a475ef658412f1d8a8a8edf65884bb41b0348b896938a353873
SimHash 4654cfc1f610

Groups

dotbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

voltron

Rule Path
Disallow /

feedburner/1.0 (http://www.feedburner.com)

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

getintentcrawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

vegi bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

adbeat_bot

Rule Path
Disallow /

jooblebot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

megaindex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

*

Rule Path
Disallow /chart_data
Disallow /chart_data/*
Disallow /api/*
Disallow /teams/*
Disallow /accept_cookies

blexbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

checkmarknetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)

Rule Path
Disallow /
Back to top

Comments

  • Hi. Questions? Email info@chartable.com
  • moz
  • 80Legs crawler
  • Somehow being used to fetch user profiles?
  • Some weird backlink checker (webmeup.com)
  • Another backlink crawler
  • Another backlink crawler (https://ahrefs.com/robot)
  • Another one.
  • don't scrape me bro
  • charts
  • Dashboard
  • Etc
  • yes
Back to top