posvudusha.com
robots.txt

Robots Exclusion Standard data for posvudusha.com

Resource Scan

Scan Details

Site Domain posvudusha.com
Base Domain posvudusha.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-03-14T11:09:03+00:00
Next Scan 2024-06-12T11:09:03+00:00

Last Successful Scan

Scanned2022-07-18T09:39:41+00:00
URL https://posvudusha.com/robots.txt
Response IP 23.224.113.29
Found Yes
Hash fde97912b37ebdd6eba20bd430538d18db7c0ef0e96475c19d6d11f3b55f8ecb
SimHash f7b1d4fcc639

Groups

*

Rule Path
Disallow /links/
Disallow /dir/
Disallow /shipping-policy
Disallow /returns-refunds-policy
Disallow /privacy-policy
Disallow /terms-of-service
Disallow /payment
Disallow /about-us
Disallow /cookies-policy
Disallow /home/
Disallow /admin/*
Disallow /enable-cookies
Disallow /index.php/
Disallow /catalog/
Disallow /livechat/
Disallow /staging/
Disallow /review/
Disallow /catalogsearch/
Disallow /newsletter/
Disallow /customer/
Disallow /orders/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customize/
Disallow /poll/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /showroom/
Disallow /downloadable/
Disallow /cart/*
Disallow /login/*
Disallow /dashboard/*
Disallow /sales/
Disallow /cart/add/
Disallow /search
Disallow /recently/*
Disallow /tapon/*
Disallow /logout
Disallow /page/*
Disallow /accounts/
Disallow /checkout/*
Disallow /checkout_shipping/
Disallow /checkout/pp/
Disallow /captcha/refresh/
Disallow /directory/currency/switch/currency/
Disallow /*.php
Disallow /sd/

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

aherfs

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

facebookexternalhit/1.1

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

experibot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

bingbot

Rule Path
Allow /

dingtalkbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

newspaper/

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100