Log In Sign Up

sipyet.com
robots.txt

Robots Exclusion Standard data for sipyet.com

Resource Scan

Scan Details

Site Domain sipyet.com
Base Domain sipyet.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-06T18:23:16+00:00
Next Scan 2025-09-20T18:23:16+00:00

Last Successful Scan

Scanned2025-08-22T18:22:36+00:00
URL http://sipyet.com/robots.txt
Domain IPs 38.63.160.22
Response IP 38.63.160.22
Found Yes
Hash 661c86d34014caf52bd0225c229d5b795cad7b3331d380caee1073ee50c952c1
SimHash 5935b16bd12a

Groups

semrushbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexa

Rule Path
Disallow /

toolbar

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

applebot

Rule Path
Disallow /

asktbfxtv

Rule Path
Disallow /

chinasospider

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

coolpadwebkit

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

curl

Rule Path
Disallow /

digext

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

elefent

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

feedly

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

indy

Rule Path
Disallow /

library

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

java

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

jorgee

Rule Path
Disallow /

lightdeckreports

Rule Path
Disallow /

bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

microsoft

Rule Path
Disallow /

url

Rule Path
Disallow /

control

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

obot

Rule Path
Disallow /

perl

Rule Path
Disallow /

psbot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

python

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spiderman

Rule Path
Disallow /

swebot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

turnitinbot-agent

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

wangidspider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

webdup

Rule Path
Disallow /

wget

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wsanalyzer

Rule Path
Disallow /

xbfmozilla

Rule Path
Disallow /

xenu

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yottaa

Rule Path
Disallow /

yunguance

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms.bot

Rule Path
Disallow /
Back to top