Log In Sign Up

anniecloth.com
robots.txt

Robots Exclusion Standard data for anniecloth.com

Resource Scan

Scan Details

Site Domain anniecloth.com
Base Domain anniecloth.com
Scan Status Ok
Last Scan2024-05-26T07:36:34+00:00
Next Scan 2024-06-02T07:36:34+00:00

Last Scan

Scanned2024-05-26T07:36:34+00:00
URL https://anniecloth.com/robots.txt
Domain IPs 108.157.254.17, 108.157.254.32, 108.157.254.39, 108.157.254.60, 2600:9000:2753:1600:17:6fca:f880:93a1, 2600:9000:2753:4800:17:6fca:f880:93a1, 2600:9000:2753:5c00:17:6fca:f880:93a1, 2600:9000:2753:6000:17:6fca:f880:93a1, 2600:9000:2753:d200:17:6fca:f880:93a1, 2600:9000:2753:e800:17:6fca:f880:93a1, 2600:9000:2753:f800:17:6fca:f880:93a1, 2600:9000:2753:fa00:17:6fca:f880:93a1
Response IP 108.157.254.17
Found Yes
Hash 42355d878e8326a1bdb800d07883bf8ffd18f997341fc8b345bda9c2b402aa86
SimHash e1f956de8611

Groups

*

Rule Path
Disallow /cart/
Disallow /checkout/
Disallow /account/
Disallow /admin/
Allow /sitemap.xml

adsbot-google

Rule Path
Disallow /admin/

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

sogou web spider/4.0

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

booch

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

doc

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

downloader

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

fetch

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

htdig

Rule Path
Disallow /

httrack

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

libwww

Rule Path
Disallow /

linko

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

psbot

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

scspider

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

soft411

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

tamu

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

xenu

Rule Path
Disallow /

zao

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

zyborg

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://anniecloth.com/sitemap.xml
Back to top