decripto.org
robots.txt
Robots Exclusion Standard data for decripto.org
Resource Scan
Scan Details
| Site Domain | decripto.org |
| Base Domain | decripto.org |
| Scan Status | Ok |
| Last Scan | 2024-11-13T13:17:17+00:00 |
| Next Scan | 2024-11-20T13:17:17+00:00 |
Last Scan
| Scanned | 2024-11-13T13:17:17+00:00 |
| URL | https://decripto.org/robots.txt |
| Domain IPs | 104.26.0.87, 104.26.1.87, 172.67.74.126, 2606:4700:20::681a:157, 2606:4700:20::681a:57, 2606:4700:20::ac43:4a7e |
| Response IP | 104.26.1.87 |
| Found | Yes |
| Hash | 31edaeb6ce515a25a15a49116d7f5f823240a4e40ce3f1527eb0ce5e0e7e79f5 |
| SimHash | 535d53aa47b7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /author/ |
| Disallow | /search/ |
| Disallow | /plugins/ |
| Disallow | /cgi-bin/ |
| Disallow | /wp-admin/ |
| Disallow | */?gcb= |
| Disallow | /*wptelegram* |
| Disallow | *?gcb=* |
| Disallow | */?fb_comment_id= |
| Disallow | *?cb=* |
| Disallow | */?s=* |
| Disallow | */wl?tme= |
| Disallow | */?utm_source= |
| Disallow | */?utm_content= |
| Disallow | */share/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
ahrefsbot
blexbot
dotbot
mj12bot
proximic
rogerbot
yandexmetrika
xenu's
sistrix
seokicks
seokicks-robot
yisouspider
qwantify
uptimebot
linespider
barkrowler
| Rule | Path |
|---|---|
| Disallow | / |
ruby
libwww-perl
dotbot
mail.ru_bot
alexibot
aqua_products
asterias
b2w
backdoorbot
becomebot
bloglovin
blowfish
bookmark search tool
bot
botalot
builtbottough
bullseye
bunnyslippers
calculon spider
cheesebot
cherrypicker
cherrypickerelite
cherrypickerse
coccoc
copernic
copyrightcheck
cosmos
crescent
daum
dittospyder
dumbot
emailcollector
emailsiphon
emailwolf
enterprise_search
erocrawler
es
exabot
extractorpro
ezooms
fairad client
fatbot
flaming attackbot
foobot
freefind
gaisbot
getright
grub
grub-client
harvest
hatena antenna
hloader
httplib
humanlinks
idg
infonavirobot
inoreader.com
iron33
jennybot
jetbot
jetbot
jikespider
kenjin spider
keyword density
larbin
lexibot
libweb/clshttp
linkextractorpro
linkscan
linkwalker
lnspiderguy
lwp-trivial
mata hari
megaindex.ru
microsoft url control
miixpc
mister pix
moget
megaindex.ru
spbot
msiecrawler
naver
netants
netmechanic
nicerspro
nutch
offline explorer
omniexplorer_bot
openbot
openfind
openfind data gathere
optimizer
oracle ultra search
parser
paperlibot
pcore-http
perman
php
propowerbot
prowebwalker
psbot
pu_in crawler
qwantify
queryn metasearch
radiation retriever
repomonkey
repomonkey bait & tackle
rma
searchpreview
seznambot
sitebot
sitesnagger
smtbot
sogou
sootle
sosospider
spankbot
spanner
spinn3r
stanford
stanford comp sci
suzuran
swiftbot
szukacz
teleport
teleportpro
telesoft
the intraformant
thenomad
tocrawl
trident
true_robot
turingos
updownerbot
url control
url_spider_pro
urly warning
vci
vci webviewer vci webviewer win32
voilabot
web image collector
webauto
webbandit
webcopier
webenhancer
websauger
website quester
webster pro
webstripper
webvac
webzip
wget
willybot
wordpress
www-collector-e
yodaobot
yak
zeus
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://decripto.org/sitemap_index.xml |
Warnings
- 1 invalid line.
Comments