cis.ieee.org
robots.txt

Robots Exclusion Standard data for cis.ieee.org

Resource Scan

Scan Details

Site Domain cis.ieee.org
Base Domain ieee.org
Scan Status Ok
Last Scan2024-09-20T14:48:08+00:00
Next Scan 2024-10-20T14:48:08+00:00

Last Scan

Scanned2024-09-20T14:48:08+00:00
URL https://cis.ieee.org/robots.txt
Domain IPs 18.235.91.131
Response IP 18.235.91.131
Found Yes
Hash f880a30150803e9b3d09f93a88924d6c30e6c93fa9748230158a933ec9ae55da
SimHash 7d35df50982f

Groups

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /templates/
Disallow /tmp/

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

chatgpt

Rule Path
Disallow /

openai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

youbot

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

spbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /
Disallow /

python-requests/2.24.0

Rule Path
Disallow /

dotbot/1.1

Rule Path
Disallow /

aria2/1.34.0

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bomborabot

Rule Path
Disallow /

rytebot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

_zbot

Rule Path
Disallow /

adsbot/3.1

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

anderspinkbot

Rule Path
Disallow /

atomseobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

cltbot

Rule Path
Disallow /

chatgpt

Rule Path
Disallow /

checkbot

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

citation_bot

Rule Path
Disallow /

codegetterbot

Rule Path
Disallow /

crowdtanglebot

Rule Path
Disallow /

crsspxlbot

Rule Path
Disallow /

cuberssreader

Rule Path
Disallow /

danibot

Rule Path
Disallow /

dark_matter_bot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

deskyobot

Rule Path
Disallow /

diffeobot

Rule Path
Disallow /

dingtalkbot-linkservice

Rule Path
Disallow /

discordbot

Rule Path
Disallow /

dubbotbot

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

folkbot

Rule Path
Disallow /

fuze_bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

getdigestbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

girafabot

Rule Path
Disallow /

gnowitnewsbot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

hatenablog-bot

Rule Path
Disallow /

hotjava

Rule Path
Disallow /

infotigerbot

Rule Path
Disallow /

java-http-client

Rule Path
Disallow /

linkisbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

livelapbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

magibot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mediumbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

monsidobot

Rule Path
Disallow /

moodlebot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

netpeakcheckerbot

Rule Path
Disallow /

nimbostratus-bot

Rule Path
Disallow /

onbbot

Rule Path
Disallow /

onalyticabot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

phractusbot

Rule Path
Disallow /

pleroma

Rule Path
Disallow /

quora-bot

Rule Path
Disallow /

rasabot

Rule Path
Disallow /

redirectbot

Rule Path
Disallow /

refindbot

Rule Path
Disallow /

researchbot

Rule Path
Disallow /

rightintelbot

Rule Path
Disallow /

riverbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serptimizerbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

solofield

Rule Path
Disallow /

sabsimbot

Rule Path
Disallow /

scribbr-citation-bot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serendeputybot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

siteanalyzerbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

sitecheckerbotcrawler

Rule Path
Disallow /

slack-imgproxy

Rule Path
Disallow /

sottopop

Rule Path
Disallow /

superbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

synologychatbot

Rule Path
Disallow /

tmmbot

Rule Path
Disallow /

tsmbot

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

terkkobot

Rule Path
Disallow /

tineye-bot

Rule Path
Disallow /

tkbot

Rule Path
Disallow /

triplecheckerrobot

Rule Path
Disallow /

tulipchain

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

v-bot

Rule Path
Disallow /

webbot

Rule Path
Disallow /

wellknownbot

Rule Path
Disallow /

xlinkbotreverter

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexrenderresourcesbot

Rule Path
Disallow /

yandexnews

Rule Path
Disallow /

yandexrca

Rule Path
Disallow /

yellowbrandprotectionbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

awesome_bot

Rule Path
Disallow /

badbot

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

bnf.fr_bot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

cubebot

Rule Path
Disallow /

curl-all

Rule Path
Disallow /

digitalshadowsbot

Rule Path
Disallow /

equellaurlbot

Rule Path
Disallow /

harsilbot

Rule Path
Disallow /

i2kconnect

Rule Path
Disallow /

isec_bot

Rule Path
Disallow /

libcurl

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

moatbot

Rule Path
Disallow /

monitorapp-robot-parser

Rule Path
Disallow /

notestock

Rule Path
Disallow /

online-webceo-bot

Rule Path
Disallow /

opengraph-bot

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /

prophy-bot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

repology-linkchecker

Rule Path
Disallow /

sc_bot

Rule Path
Disallow /

seo-audit-check-bot

Rule Path
Disallow /

startmebot

Rule Path
Disallow /

strutbot

Rule Path
Disallow /

trendictionbot0

Rule Path
Disallow /

undefined

Rule Path
Disallow /

unirest-java

Rule Path
Disallow /

urlchecker

Rule Path
Disallow /

yandexaccessibilitybot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

python

Rule Path
Disallow /

Comments

  • Blocking specific unwanted user agents
  • Blocking known AI systems
  • Additional unwanted bots
  • Custombot sigs

Warnings

  • 34 invalid lines.
  • `ser-agent` is not a known field.