Log In Sign Up

gradecalc.info
robots.txt

Robots Exclusion Standard data for gradecalc.info

Resource Scan

Scan Details

Site Domain gradecalc.info
Base Domain gradecalc.info
Scan Status Ok
Last Scan2024-10-04T07:02:16+00:00
Next Scan 2024-10-11T07:02:16+00:00

Last Scan

Scanned2024-10-04T07:02:16+00:00
URL https://gradecalc.info/robots.txt
Domain IPs 100.29.179.246, 54.145.217.86
Response IP 54.145.217.86
Found Yes
Hash 95a2c088a9fad1f7c73159553e756971bd2e8fcf0ac16583effc3e50cfd8f268
SimHash 7204495b8c20

Groups

adbeat_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

deusu

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

extlinksbot

Rule Path
Disallow /

getintent crawler

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

hyscore

Rule Path
Disallow /

httrack

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mappy

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

obot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

seeker

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

veoozbot

Rule Path
Disallow /

virusdie crawler

Rule Path
Disallow /

wf search

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

yeti

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Comments

  • hxxps://www.adbeat.com/operation_policy
  • hxxps://ahrefs.com/robot
  • hxxp://alphaseobot.com/bot.html
  • hxxps://archive.org/details/archive.org_bot
  • hxxps://awario.com/bots.html
  • hxxp://webmeup-crawler.com/
  • hxxp://law.di.unimi.it/BUbiNG.html
  • hxxp://commoncrawl.org/big-picture/frequently-asked-questions/
  • hxxps://cliqz.com/en/cliqzbot
  • hxxp://help.coccoc.com/en/search-engine
  • hxxps://deusu.de/robot.html
  • hxxp://domainstats.io/our-bot
  • hxxp://moz.com/products
  • hxxp://www.opensiteexplorer.org/dotbot
  • hxxps://www.exalead.com/search/webmasterguide
  • hxxps://extlinks.com/Bot.html
  • hxxps://getintent.com/bot.html
  • hxxp://www.grapeshot.com/crawler/
  • hxxps://hyscore.io/crawler/
  • hxxps://integralads.com/site-indexing-policy/
  • hxxps://cognitiveseo.com/bot.html
  • hxxps://www.linguee.com/bot
  • hxxps://help.mail.ru/webmaster/indexing/robots.txt/rules/user-agent
  • hxxp://mappydata.net/
  • hxxp://mj12bot.com/
  • hxxps://www.mojeek.com/bot.html
  • hxxps://neeva.com/neevabot
  • hxxps://www.netseer.com/crawler/
  • hxxp://filterdb.iss.net/crawler/
  • hxxps://webmaster.petalsearch.com/site/petalbot
  • hxxps://www.comscore.com/proximic-spider
  • hxxps://www.safedns.com/en/searchbot/
  • hxxp://www.scoutjet.com/
  • hxxp://lookseek.com/seeker/
  • hxxps://bot.seekport.com/
  • hxxps://www.seokicks.de/robot.html
  • To block SEMrushBot from crawling your site for web graph of links, add:
  • To remove SEMrushBot from crawling your site for different SEO and technical issues, add:
  • hxxps://napoveda.seznam.cz/en/full-text-search/crawling-control/
  • hxxps://siteexplorer.info/Backlink-Checker-Spider/
  • hxxps://www.similartech.com/smtbot
  • hxxp://openlinkprofiler.org/bot
  • hxxp://sur.ly/bot.html
  • hxxp://www.trendiction.com/en/publisher/bot
  • hxxps://turnitin.com/robot/crawlerinfo.html
  • hxxp://www.veooz.com/veoozbot.html
  • hxxp://www.wotbox.com/bot/
  • hxxps://help.naver.com/support/contents/contents.nhn?serviceNo=19634&categoryNo=19668
  • hxxps://suite.seozoom.it/bot.html
Back to top