Log In Sign Up

annunci69.it
robots.txt

Robots Exclusion Standard data for annunci69.it

Resource Scan

Scan Details

Site Domain annunci69.it
Base Domain annunci69.it
Scan Status Ok
Last Scan2024-10-21T09:48:04+00:00
Next Scan 2024-11-20T09:48:04+00:00

Last Scan

Scanned2024-10-21T09:48:04+00:00
URL https://annunci69.it/robots.txt
Redirect https://www.annunci69.it/robots.txt
Redirect Domain www.annunci69.it
Redirect Base annunci69.it
Domain IPs 104.23.139.17, 104.23.140.17
Redirect IPs 104.23.139.17, 104.23.140.17
Response IP 104.23.140.17
Found Yes
Hash 9547778fb3ea6fb06e4894523dd49e21530976afeadac48511d97fd397445ce5
SimHash f39f5262c9e7

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /action/

botrighthere

Rule Path
Disallow

webzip

Rule Path
Disallow

larbin

Rule Path
Disallow

b2w/0.1

Rule Path
Disallow

copernic

Rule Path
Disallow

psbot

Rule Path
Disallow

python-urllib

Rule Path
Disallow

url_spider_pro

Rule Path
Disallow

cherrypicker

Rule Path
Disallow

emailcollector

Rule Path
Disallow

emailsiphon

Rule Path
Disallow

webbandit

Rule Path
Disallow

emailwolf

Rule Path
Disallow

extractorpro

Rule Path
Disallow

copyrightcheck

Rule Path
Disallow

crescent

Rule Path
Disallow

sitesnagger

Rule Path
Disallow

prowebwalker

Rule Path
Disallow

cheesebot

Rule Path
Disallow

lnspiderguy

Rule Path
Disallow

alexibot

Rule Path
Disallow

teleport

Rule Path
Disallow

teleportpro

Rule Path
Disallow

miixpc

Rule Path
Disallow

telesoft

Rule Path
Disallow

website quester

Rule Path
Disallow

webzip

Rule Path
Disallow

moget/2.1

Rule Path
Disallow

webzip/4.0

Rule Path
Disallow

webstripper

Rule Path
Disallow

websauger

Rule Path
Disallow

webcopier

Rule Path
Disallow

netants

Rule Path
Disallow

mister pix

Rule Path
Disallow

webauto

Rule Path
Disallow

thenomad

Rule Path
Disallow

www-collector-e

Rule Path
Disallow

rma

Rule Path
Disallow

libweb/clshttp

Rule Path
Disallow

asterias

Rule Path
Disallow

httplib

Rule Path
Disallow

turingos

Rule Path
Disallow

spanner

Rule Path
Disallow

infonavirobot

Rule Path
Disallow

harvest/1.5

Rule Path
Disallow

bullseye/1.0

Rule Path
Disallow

mozilla/4.0 (compatible; bullseye; windows 95)

Rule Path
Disallow

crescent internet toolpak http ole control v.1.0

Rule Path
Disallow

cherrypickerse/1.0

Rule Path
Disallow

cherrypickerelite/1.0

Rule Path
Disallow

webbandit/3.50

Rule Path
Disallow

nicerspro

Rule Path
Disallow

microsoft url control - 5.01.4511

Rule Path
Disallow

dittospyder

Rule Path
Disallow

foobot

Rule Path
Disallow

spankbot

Rule Path
Disallow

botalot

Rule Path
Disallow

lwp-trivial/1.34

Rule Path
Disallow

lwp-trivial

Rule Path
Disallow

bunnyslippers

Rule Path
Disallow

microsoft url control - 6.00.8169

Rule Path
Disallow

urly warning

Rule Path
Disallow

wget/1.6

Rule Path
Disallow

wget/1.5.3

Rule Path
Disallow

wget

Rule Path
Disallow

linkwalker

Rule Path
Disallow

cosmos

Rule Path
Disallow

moget

Rule Path
Disallow

hloader

Rule Path
Disallow

humanlinks

Rule Path
Disallow

linkextractorpro

Rule Path
Disallow

offline explorer

Rule Path
Disallow

mata hari

Rule Path
Disallow

lexibot

Rule Path
Disallow

web image collector

Rule Path
Disallow

the intraformant

Rule Path
Disallow

true_robot/1.0

Rule Path
Disallow

true_robot

Rule Path
Disallow

blowfish/1.0

Rule Path
Disallow

jennybot

Rule Path
Disallow

miixpc/4.2

Rule Path
Disallow

builtbottough

Rule Path
Disallow

propowerbot/2.14

Rule Path
Disallow

backdoorbot/1.0

Rule Path
Disallow

tocrawl/urldispatcher

Rule Path
Disallow

webenhancer

Rule Path
Disallow

suzuran

Rule Path
Disallow

tighttwatbot

Rule Path
Disallow

vci webviewer vci webviewer win32

Rule Path
Disallow

vci

Rule Path
Disallow

szukacz/1.4

Rule Path
Disallow

queryn metasearch

Rule Path
Disallow

openfind data gatherer

Rule Path
Disallow

openfind

Rule Path
Disallow

xenu's link sleuth 1.1c

Rule Path
Disallow

xenu's

Rule Path
Disallow

zeus

Rule Path
Disallow

repomonkey bait & tackle/v1.01

Rule Path
Disallow

repomonkey

Rule Path
Disallow

microsoft url control

Rule Path
Disallow

openbot

Rule Path
Disallow

url control

Rule Path
Disallow

zeus link scout

Rule Path
Disallow

zeus 32297 webster pro v2.9 win32

Rule Path
Disallow

webster pro

Rule Path
Disallow

erocrawler

Rule Path
Disallow

linkscan/8.1a unix

Rule Path
Disallow

keyword density/0.9

Rule Path
Disallow

kenjin spider

Rule Path
Disallow

iron33/1.0.2

Rule Path
Disallow

bookmark search tool

Rule Path
Disallow

getright/4.2

Rule Path
Disallow

fairad client

Rule Path
Disallow

gaisbot

Rule Path
Disallow

aqua_products

Rule Path
Disallow

radiation retriever 1.1

Rule Path
Disallow

flaming attackbot

Rule Path
Disallow

oracle ultra search

Rule Path
Disallow

msiecrawler

Rule Path
Disallow

perman

Rule Path
Disallow

searchpreview

Rule Path
Disallow

turnitinbot

Rule Path
Disallow

wget

Rule Path
Disallow

extractorpro

Rule Path
Disallow

webzip/4.21

Rule Path
Disallow

webzip/5.0

Rule Path
Disallow

httrack 3.0

Rule Path
Disallow

turnitinbot/1.5

Rule Path
Disallow

webcopier v3.2a

Rule Path
Disallow

webcapture 2.0

Rule Path
Disallow

webcopier v.2.2

Rule Path
Disallow
Back to top