Log In Sign Up

cbrfairing.com
robots.txt

Robots Exclusion Standard data for cbrfairing.com

Resource Scan

Scan Details

Site Domain cbrfairing.com
Base Domain cbrfairing.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-08-11T07:18:26+00:00
Next Scan 2024-11-09T07:18:26+00:00

Last Successful Scan

Scanned2023-03-26T04:49:24+00:00
URL https://cbrfairing.com/robots.txt
Domain IPs 190.92.133.2
Response IP 190.92.133.2
Found Yes
Hash b68734fa6e75a19ecf3017074f733ffc8228d418a78a9069fef6ceb58a907740
SimHash 766117dec631

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 20

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot

Rule Path
Disallow /index.php?route=product%2Fcatalog
Disallow /index.php?route=product%2Fcompare
Disallow /index.php?route=account%2Freward
Disallow /index.php?route=checkout%2Fcart
Disallow /index.php?route=information%2Fcontact
Disallow /index.php?route=account%2Freturn%2Fadd
Disallow /index.php?route=information%2Fcontact
Disallow /index.php?route=account%2Fwishlist
Disallow /privacy
Disallow /terms
Back to top

Warnings

  • 1 invalid line.
Back to top