/.well-known/

Log In Sign Up

fuckgfw.pw
robots.txt

Robots Exclusion Standard data for fuckgfw.pw

Archived Snapshots

Resource Scan

Scan Details

Site Domain	fuckgfw.pw
Base Domain	fuckgfw.pw
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Server returned a server error.
Last Scan	2024-11-16T06:43:59+00:00
Next Scan	2024-11-30T06:43:59+00:00

Last Successful Scan

Scanned	2024-11-01T06:43:13+00:00
URL	https://www.fuckgfw.pw/robots.txt
Domain IPs	104.21.15.205, 172.67.164.56, 2606:4700:3031::6815:fcd, 2606:4700:3036::ac43:a438
Response IP	172.67.164.56
Found	Yes
Hash	ff30a4b076407d90de44caae7893e8d6d4ca67590d8f5fba6913d09d31932d2d
SimHash	63436d732bef

Groups

abonti

Rule

Path

Disallow

/

aboundex

Rule

Path

Disallow

/

acunetix

Rule

Path

Disallow

/

admantx

Rule

Path

Disallow

/

afd-verbotsverfahren

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

aibot

Rule

Path

Disallow

/

aihitbot

Rule

Path

Disallow

/

aipbot

Rule

Path

Disallow

/

alexibot

Rule

Path

Disallow

/

alligator

Rule

Path

Disallow

/

allsubmitter

Rule

Path

Disallow

/

alphabot

Rule

Path

Disallow

/

anarchie

Rule

Path

Disallow

/

apexoo

Rule

Path

Disallow

/

archive.org_bot

Rule

Path

Disallow

/

aspseek

Rule

Path

Disallow

/

asterias

Rule

Path

Disallow

/

attach

Rule

Path

Disallow

/

autoemailspider

Rule

Path

Disallow

/

backdoorbot

Rule

Path

Disallow

/

backlink-ceck

Rule

Path

Disallow

/

backlink-check

Rule

Path

Disallow

/

backlinkcrawler

Rule

Path

Disallow

/

backstreet

Rule

Path

Disallow

/

backweb

Rule

Path

Disallow

/

badass

Rule

Path

Disallow

/

baiduspider

Rule

Path

Disallow

/

bandit

Rule

Path

Disallow

/

barkrowler

Rule

Path

Disallow

/

batchftp

Rule

Path

Disallow

/

battleztar bazinga

Rule

Path

Disallow

/

bbbike

Rule

Path

Disallow

/

bdcbot

Rule

Path

Disallow

/

bdfetch

Rule

Path

Disallow

/

betabot

Rule

Path

Disallow

/

bigfoot

Rule

Path

Disallow

/

bitacle

Rule

Path

Disallow

/

blackboard

Rule

Path

Disallow

/

black hole

Rule

Path

Disallow

/

blackwidow

Rule

Path

Disallow

/

blexbot

Rule

Path

Disallow

/

blow

Rule

Path

Disallow

/

blowfish

Rule

Path

Disallow

/

boardreader

Rule

Path

Disallow

/

bolt

Rule

Path

Disallow

/

botalot

Rule

Path

Disallow

/

brandprotect

Rule

Path

Disallow

/

brandwatch

Rule

Path

Disallow

/

bubing

Rule

Path

Disallow

/

buddy

Rule

Path

Disallow

/

builtbottough

Rule

Path

Disallow

/

builtwith

Rule

Path

Disallow

/

bullseye

Rule

Path

Disallow

/

bunnyslippers

Rule

Path

Disallow

/

buzzsumo

Rule

Path

Disallow

/

calculon

Rule

Path

Disallow

/

catexplorador

Rule

Path

Disallow

/

cazoodlebot

Rule

Path

Disallow

/

ccbot

Rule

Path

Disallow

/

cegbfeieh

Rule

Path

Disallow

/

cheesebot

Rule

Path

Disallow

/

cherrypicker

Rule

Path

Disallow

/

cheteam

Rule

Path

Disallow

/

chinaclaw

Rule

Path

Disallow

/

chlooe

Rule

Path

Disallow

/

claritybot

Rule

Path

Disallow

/

cliqzbot

Rule

Path

Disallow

/

cloud mapping

Rule

Path

Disallow

/

coccocbot-web

Rule

Path

Disallow

/

cogentbot

Rule

Path

Disallow

/

cognitiveseo

Rule

Path

Disallow

/

collector

Rule

Path

Disallow

/

com.plumanalytics

Rule

Path

Disallow

/

copier

Rule

Path

Disallow

/

copyrightcheck

Rule

Path

Disallow

/

copyscape

Rule

Path

Disallow

/

cosmos

Rule

Path

Disallow

/

craftbot

Rule

Path

Disallow

/

crawler4j

Rule

Path

Disallow

/

crawler.feedback

Rule

Path

Disallow

/

crawl.sogou.com

Rule

Path

Disallow

/

crazywebcrawler

Rule

Path

Disallow

/

crescent

Rule

Path

Disallow

/

cshttp

Rule

Path

Disallow

/

curious

Rule

Path

Disallow

/

custo

Rule

Path

Disallow

/

databasedrivermysqli

Rule

Path

Disallow

/

datacha0s

Rule

Path

Disallow

/

dblbot

Rule

Path

Disallow

/

demandbase-bot

Rule

Path

Disallow

/

demon

Rule

Path

Disallow

/

deusu

Rule

Path

Disallow

/

devil

Rule

Path

Disallow

/

digincore

Rule

Path

Disallow

/

digitalpebble

Rule

Path

Disallow

/

diibot

Rule

Path

Disallow

/

dirbuster

Rule

Path

Disallow

/

disco

Rule

Path

Disallow

/

discobot

Rule

Path

Disallow

/

discoverybot

Rule

Path

Disallow

/

dittospyder

Rule

Path

Disallow

/

dnyzbot

Rule

Path

Disallow

/

domainappender

Rule

Path

Disallow

/

domaincrawler

Rule

Path

Disallow

/

domainsigmacrawler

Rule

Path

Disallow

/

domainstatsbot

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

download wonder

Rule

Path

Disallow

/

dragonfly

Rule

Path

Disallow

/

drip

Rule

Path

Disallow

/

dts agent

Rule

Path

Disallow

/

easydl

Rule

Path

Disallow

/

ebingbong

Rule

Path

Disallow

/

ecatch

Rule

Path

Disallow

/

eccp/1.0

Rule

Path

Disallow

/

ecxi

Rule

Path

Disallow

/

eirgrabber

Rule

Path

Disallow

/

email siphon

Rule

Path

Disallow

/

email wolf

Rule

Path

Disallow

/

erocrawler

Rule

Path

Disallow

/

evc-batch

Rule

Path

Disallow

/

evil

Rule

Path

Disallow

/

exabot

Rule

Path

Disallow

/

express webpictures

Rule

Path

Disallow

/

extlinksbot

Rule

Path

Disallow

/

extractor

Rule

Path

Disallow

/

extractorpro

Rule

Path

Disallow

/

extreme picture finder

Rule

Path

Disallow

/

eyenetie

Rule

Path

Disallow

/

ezooms

Rule

Path

Disallow

/

fdm

Rule

Path

Disallow

/

femtosearchbot

Rule

Path

Disallow

/

fhscan

Rule

Path

Disallow

/

fimap

Rule

Path

Disallow

/

firefox/7.0

Rule

Path

Disallow

/

flashget

Rule

Path

Disallow

/

flunky

Rule

Path

Disallow

/

foobot

Rule

Path

Disallow

/

freeuploader

Rule

Path

Disallow

/

frontpage

Rule

Path

Disallow

/

fyberspider

Rule

Path

Disallow

/

fyrebot

Rule

Path

Disallow

/

galaxybot

Rule

Path

Disallow

/

genieo

Rule

Path

Disallow

/

germcrawler

Rule

Path

Disallow

/

getintent

Rule

Path

Disallow

/

getright

Rule

Path

Disallow

/

getweb

Rule

Path

Disallow

/

gigablast

Rule

Path

Disallow

/

gigabot

Rule

Path

Disallow

/

g-i-g-a-b-o-t

Rule

Path

Disallow

/

go-ahead-got-it

Rule

Path

Disallow

/

gotit

Rule

Path

Disallow

/

gozilla

Rule

Path

Disallow

/

go!zilla

Rule

Path

Disallow

/

grabber

Rule

Path

Disallow

/

grabnet

Rule

Path

Disallow

/

grafula

Rule

Path

Disallow

/

grapefx

Rule

Path

Disallow

/

grapeshotcrawler

Rule

Path

Disallow

/

gridbot

Rule

Path

Disallow

/

gt::www

Rule

Path

Disallow

/

haansoft

Rule

Path

Disallow

/

haosouspider

Rule

Path

Disallow

/

harvest

Rule

Path

Disallow

/

havij

Rule

Path

Disallow

/

headmasterseo

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

hloader

Rule

Path

Disallow

/

hmview

Rule

Path

Disallow

/

htmlparser

Rule

Path

Disallow

/

http::lite

Rule

Path

Disallow

/

httrack

Rule

Path

Disallow

/

humanlinks

Rule

Path

Disallow

/

hybridbot

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

iblog

Rule

Path

Disallow

/

idbot

Rule

Path

Disallow

/

id-search

Rule

Path

Disallow

/

ilsebot

Rule

Path

Disallow

/

image fetch

Rule

Path

Disallow

/

image sucker

Rule

Path

Disallow

/

indeedbot

Rule

Path

Disallow

/

indy library

Rule

Path

Disallow

/

infonavirobot

Rule

Path

Disallow

/

infotekies

Rule

Path

Disallow

/

instabid

Rule

Path

Disallow

/

intelliseek

Rule

Path

Disallow

/

interget

Rule

Path

Disallow

/

internet ninja

Rule

Path

Disallow

/

internetseer

Rule

Path

Disallow

/

internetvista monitor

Rule

Path

Disallow

/

ips-agent

Rule

Path

Disallow

/

iria

Rule

Path

Disallow

/

irlbot

Rule

Path

Disallow

/

iskanie

Rule

Path

Disallow

/

istellabot

Rule

Path

Disallow

/

jamesbot

Rule

Path

Disallow

/

jbrofuzz

Rule

Path

Disallow

/

jennybot

Rule

Path

Disallow

/

jetcar

Rule

Path

Disallow

/

jikespider

Rule

Path

Disallow

/

joc web spider

Rule

Path

Disallow

/

joomla

Rule

Path

Disallow

/

jorgee

Rule

Path

Disallow

/

justview

Rule

Path

Disallow

/

jyxobot

Rule

Path

Disallow

/

kenjin spider

Rule

Path

Disallow

/

keyword density

Rule

Path

Disallow

/

kozmosbot

Rule

Path

Disallow

/

lanshanbot

Rule

Path

Disallow

/

larbin

Rule

Path

Disallow

/

leechftp

Rule

Path

Disallow

/

leechget

Rule

Path

Disallow

/

lexibot

Rule

Path

Disallow

/

lftp

Rule

Path

Disallow

/

libweb

Rule

Path

Disallow

/

libwhisker

Rule

Path

Disallow

/

lightspeedsystems

Rule

Path

Disallow

/

likse

Rule

Path

Disallow

/

linguee

Rule

Path

Disallow

/

linkdexbot

Rule

Path

Disallow

/

linkextractorpro

Rule

Path

Disallow

/

linkpadbot

Rule

Path

Disallow

/

linkscan

Rule

Path

Disallow

/

linksmanager

Rule

Path

Disallow

/

linkwalker

Rule

Path

Disallow

/

linqiametadatadownloaderbot

Rule

Path

Disallow

/

linqiarssbot

Rule

Path

Disallow

/

linqiascrapebot

Rule

Path

Disallow

/

lipperhey

Rule

Path

Disallow

/

litemage_walker

Rule

Path

Disallow

/

lmspider

Rule

Path

Disallow

/

lnspiderguy

Rule

Path

Disallow

/

ltx71

Rule

Path

Disallow

/

lwp-request

Rule

Path

Disallow

/

lwp::simple

Rule

Path

Disallow

/

lwp-trivial

Rule

Path

Disallow

/

magnet

Rule

Path

Disallow

/

mag-net

Rule

Path

Disallow

/

magpie-crawler

Rule

Path

Disallow

/

mail.ru_bot

Rule

Path

Disallow

/

majestic12

Rule

Path

Disallow

/

markmonitor

Rule

Path

Disallow

/

markwatch

Rule

Path

Disallow

/

masscan

Rule

Path

Disallow

/

mass downloader

Rule

Path

Disallow

/

mata hari

Rule

Path

Disallow

/

mauibot

Rule

Path

Disallow

/

meanpathbot

Rule

Path

Disallow

/

mediatoolkitbot

Rule

Path

Disallow

/

mediawords

Rule

Path

Disallow

/

megaindex.ru

Rule

Path

Disallow

/

metauri

Rule

Path

Disallow

/

mfc_tear_sample

Rule

Path

Disallow

/

microsoft data access

Rule

Path

Disallow

/

microsoft url control

Rule

Path

Disallow

/

midown tool

Rule

Path

Disallow

/

miixpc

Rule

Path

Disallow

/

mister pix

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

mojeek

Rule

Path

Disallow

/

morfeus fucking scanner

Rule

Path

Disallow

/

mr.4x3

Rule

Path

Disallow

/

msfrontpage

Rule

Path

Disallow

/

msiecrawler

Rule

Path

Disallow

/

msnbot

Rule

Path

Disallow

/

msrabot

Rule

Path

Disallow

/

ms web services client protocol

Rule

Path

Disallow

/

muhstik-scan

Rule

Path

Disallow

/

musobot

Rule

Path

Disallow

/

name intelligence

Rule

Path

Disallow

/

nameprotect

Rule

Path

Disallow

/

navroad

Rule

Path

Disallow

/

nearsite

Rule

Path

Disallow

/

needle

Rule

Path

Disallow

/

nessus

Rule

Path

Disallow

/

netants

Rule

Path

Disallow

/

netcraft

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

netlyzer

Rule

Path

Disallow

/

netmechanic

Rule

Path

Disallow

/

netspider

Rule

Path

Disallow

/

nettrack

Rule

Path

Disallow

/

net vampire

Rule

Path

Disallow

/

netvibes

Rule

Path

Disallow

/

netzip

Rule

Path

Disallow

/

nextgensearchbot

Rule

Path

Disallow

/

nibbler

Rule

Path

Disallow

/

nicerspro

Rule

Path

Disallow

/

niki-bot

Rule

Path

Disallow

/

nikto

Rule

Path

Disallow

/

nimblecrawler

Rule

Path

Disallow

/

ninja

Rule

Path

Disallow

/

nmap

Rule

Path

Disallow

/

npbot

Rule

Path

Disallow

/

nutch

Rule

Path

Disallow

/

obot

Rule

Path

Disallow

/

octopus

Rule

Path

Disallow

/

offline explorer

Rule

Path

Disallow

/

offline navigator

Rule

Path

Disallow

/

openfind

Rule

Path

Disallow

/

openlinkprofiler

Rule

Path

Disallow

/

openvas

Rule

Path

Disallow

/

openvas

Rule

Path

Disallow

/

orangebot

Rule

Path

Disallow

/

orangespider

Rule

Path

Disallow

/

outclicksbot

Rule

Path

Disallow

/

outfoxbot

Rule

Path

Disallow

/

pageanalyzer

Rule

Path

Disallow

/

page analyzer

Rule

Path

Disallow

/

pagegrabber

Rule

Path

Disallow

/

page scorer

Rule

Path

Disallow

/

pagescorer

Rule

Path

Disallow

/

panscient

Rule

Path

Disallow

/

papa foto

Rule

Path

Disallow

/

pavuk

Rule

Path

Disallow

/

pcbrowser

Rule

Path

Disallow

/

pecl::http

Rule

Path

Disallow

/

peoplepal

Rule

Path

Disallow

/

phpcrawl

Rule

Path

Disallow

/

picscout

Rule

Path

Disallow

/

picsearch

Rule

Path

Disallow

/

picturefinder

Rule

Path

Disallow

/

pimonster

Rule

Path

Disallow

/

pi-monster

Rule

Path

Disallow

/

pixray

Rule

Path

Disallow

/

pleasecrawl

Rule

Path

Disallow

/

plumanalytics

Rule

Path

Disallow

/

pockey

Rule

Path

Disallow

/

poe-component-client-http

Rule

Path

Disallow

/

probethenet

Rule

Path

Disallow

/

propowerbot

Rule

Path

Disallow

/

prowebwalker

Rule

Path

Disallow

/

psbot

Rule

Path

Disallow

/

pump

Rule

Path

Disallow

/

pxbroker

Rule

Path

Disallow

/

pycurl

Rule

Path

Disallow

/

queryn metasearch

Rule

Path

Disallow

/

quick-crawler

Rule

Path

Disallow

/

rankactive

Rule

Path

Disallow

/

rankactivelinkbot

Rule

Path

Disallow

/

rankflex

Rule

Path

Disallow

/

rankingbot

Rule

Path

Disallow

/

rankingbot2

Rule

Path

Disallow

/

rankivabot

Rule

Path

Disallow

/

rankurbot

Rule

Path

Disallow

/

realdownload

Rule

Path

Disallow

/

reaper

Rule

Path

Disallow

/

rebelmouse

Rule

Path

Disallow

/

recorder

Rule

Path

Disallow

/

redesscrapy

Rule

Path

Disallow

/

reget

Rule

Path

Disallow

/

repomonkey

Rule

Path

Disallow

/

ripper

Rule

Path

Disallow

/

robozilla

Rule

Path

Disallow

/

rocketcrawler

Rule

Path

Disallow

/

rogerbot

Rule

Path

Disallow

/

s1z.ru

Rule

Path

Disallow

/

salesintelligent

Rule

Path

Disallow

/

sbider

Rule

Path

Disallow

/

scanalert

Rule

Path

Disallow

/

scanbot

Rule

Path

Disallow

/

scan.lol

Rule

Path

Disallow

/

scrapy

Rule

Path

Disallow

/

screaming

Rule

Path

Disallow

/

screenerbot

Rule

Path

Disallow

/

scrubby

Rule

Path

Disallow

/

searchestate

Rule

Path

Disallow

/

searchmetricsbot

Rule

Path

Disallow

/

semrush

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

seokicks

Rule

Path

Disallow

/

seolyticscrawler

Rule

Path

Disallow

/

seomoz

Rule

Path

Disallow

/

seoprofiler

Rule

Path

Disallow

/

seoscanners

Rule

Path

Disallow

/

seositecheckup

Rule

Path

Disallow

/

seostats

Rule

Path

Disallow

/

sexsearcher

Rule

Path

Disallow

/

shodan

Rule

Path

Disallow

/

siphon

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

sitebeam

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

siteimprove

Rule

Path

Disallow

/

sitelockspider

Rule

Path

Disallow

/

sitesnagger

Rule

Path

Disallow

/

sitesucker

Rule

Path

Disallow

/

site sucker

Rule

Path

Disallow

/

sitevigil

Rule

Path

Disallow

/

slackbot-linkexpanding

Rule

Path

Disallow

/

slysearch

Rule

Path

Disallow

/

slurp

Rule

Path

Disallow

/

smartdownload

Rule

Path

Disallow

/

smtbot

Rule

Path

Disallow

/

snake

Rule

Path

Disallow

/

snapbot

Rule

Path

Disallow

/

snoopy

Rule

Path

Disallow

/

socialrankiobot

Rule

Path

Disallow

/

sogouspider

Rule

Path

Disallow

/

sogou web spider

Rule

Path

Disallow

/

sosospider

Rule

Path

Disallow

/

sottopop

Rule

Path

Disallow

/

spacebison

Rule

Path

Disallow

/

spammen

Rule

Path

Disallow

/

spankbot

Rule

Path

Disallow

/

spanner

Rule

Path

Disallow

/

spbot

Rule

Path

Disallow

/

spinn3r

Rule

Path

Disallow

/

sputnikbot

Rule

Path

Disallow

/

sqlmap

Rule

Path

Disallow

/

sqlworm

Rule

Path

Disallow

/

sqworm

Rule

Path

Disallow

/

steeler

Rule

Path

Disallow

/

stripper

Rule

Path

Disallow

/

sucker

Rule

Path

Disallow

/

sucuri

Rule

Path

Disallow

/

superbot

Rule

Path

Disallow

/

superhttp

Rule

Path

Disallow

/

surfbot

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

suzuran

Rule

Path

Disallow

/

swiftbot

Rule

Path

Disallow

/

sysscan

Rule

Path

Disallow

/

szukacz

Rule

Path

Disallow

/

t0phackteam

Rule

Path

Disallow

/

t8abot

Rule

Path

Disallow

/

takeout

Rule

Path

Disallow

/

teleport

Rule

Path

Disallow

/

teleportpro

Rule

Path

Disallow

/

telesoft

Rule

Path

Disallow

/

telesphoreo

Rule

Path

Disallow

/

telesphorep

Rule

Path

Disallow

/

teoma

Rule

Path

Disallow

/

the intraformant

Rule

Path

Disallow

/

thenomad

Rule

Path

Disallow

/

tighttwatbot

Rule

Path

Disallow

/

titan

Rule

Path

Disallow

/

toata

Rule

Path

Disallow

/

toweyabot

Rule

Path

Disallow

/

tracemyfile

Rule

Path

Disallow

/

trendiction

Rule

Path

Disallow

/

trendictionbot

Rule

Path

Disallow

/

trendiction.com

Rule

Path

Disallow

/

trendiction.de

Rule

Path

Disallow

/

true_robot

Rule

Path

Disallow

/

turingos

Rule

Path

Disallow

/

turnitin

Rule

Path

Disallow

/

turnitinbot

Rule

Path

Disallow

/

twengabot

Rule

Path

Disallow

/

twice

Rule

Path

Disallow

/

twiceler

Rule

Path

Disallow

/

typhoeus

Rule

Path

Disallow

/

unisterbot

Rule

Path

Disallow

/

urly warning

Rule

Path

Disallow

/

urly.warning

Rule

Path

Disallow

/

vacuum

Rule

Path

Disallow

/

vagabondo

Rule

Path

Disallow

/

vb project

Rule

Path

Disallow

/

vci

Rule

Path

Disallow

/

vericitecrawler

Rule

Path

Disallow

/

vidiblescraper

Rule

Path

Disallow

/

virusdie

Rule

Path

Disallow

/

voideye

Rule

Path

Disallow

/

voil

Rule

Path

Disallow

/

voltron

Rule

Path

Disallow

/

wallpapers/3.0

Rule

Path

Disallow

/

wallpapershd

Rule

Path

Disallow

/

wasalive-bot

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

webalta

Rule

Path

Disallow

/

webauto

Rule

Path

Disallow

/

web auto

Rule

Path

Disallow

/

webbandit

Rule

Path

Disallow

/

webcollage

Rule

Path

Disallow

/

web collage

Rule

Path

Disallow

/

webcopier

Rule

Path

Disallow

/

webdav

Rule

Path

Disallow

/

webenhancer

Rule

Path

Disallow

/

web enhancer

Rule

Path

Disallow

/

webfetch

Rule

Path

Disallow

/

web fetch

Rule

Path

Disallow

/

webfuck

Rule

Path

Disallow

/

web fuck

Rule

Path

Disallow

/

webgo is

Rule

Path

Disallow

/

webimagecollector

Rule

Path

Disallow

/

webleacher

Rule

Path

Disallow

/

webmasterworldforumbot

Rule

Path

Disallow

/

webmeup-crawler

Rule

Path

Disallow

/

webpix

Rule

Path

Disallow

/

web pix

Rule

Path

Disallow

/

webreaper

Rule

Path

Disallow

/

websauger

Rule

Path

Disallow

/

web sauger

Rule

Path

Disallow

/

webshag

Rule

Path

Disallow

/

websiteextractor

Rule

Path

Disallow

/

websitequester

Rule

Path

Disallow

/

website quester

Rule

Path

Disallow

/

webster

Rule

Path

Disallow

/

webstripper

Rule

Path

Disallow

/

websucker

Rule

Path

Disallow

/

web sucker

Rule

Path

Disallow

/

webwhacker

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

wesee

Rule

Path

Disallow

/

whack

Rule

Path

Disallow

/

whacker

Rule

Path

Disallow

/

whatweb

Rule

Path

Disallow

/

who.is bot

Rule

Path

Disallow

/

widow

Rule

Path

Disallow

/

winhttrack

Rule

Path

Disallow

/

wiseguys robot

Rule

Path

Disallow

/

wisenutbot

Rule

Path

Disallow

/

wonderbot

Rule

Path

Disallow

/

woobot

Rule

Path

Disallow

/

wotbox

Rule

Path

Disallow

/

wprecon

Rule

Path

Disallow

/

wpscan

Rule

Path

Disallow

/

www-collector-e

Rule

Path

Disallow

/

www-mechanize

Rule

Path

Disallow

/

www::mechanize

Rule

Path

Disallow

/

wwwoffle

Rule

Path

Disallow

/

x09mozilla

Rule

Path

Disallow

/

x22mozilla

Rule

Path

Disallow

/

xaldon webspider

Rule

Path

Disallow

/

xaldon_webspider

Rule

Path

Disallow

/

xenu

Rule

Path

Disallow

/

xpymep1.exe

Rule

Path

Disallow

/

yandexbot

Rule

Path

Disallow

/

youdaobot

Rule

Path

Disallow

/

zade

Rule

Path

Disallow

/

zauba

Rule

Path

Disallow

/

zauba.io

Rule

Path

Disallow

/

zermelo

Rule

Path

Disallow

/

zeus

Rule

Path

Disallow

/

zgrab

Rule

Path

Disallow

/

zitebot

Rule

Path

Disallow

/

zmeu

Rule

Path

Disallow

/

zoominfobot

Rule

Path

Disallow

/

zumbot

Rule

Path

Disallow

/

zyborg

Rule

Path

Disallow

/

Disallow

/

Other Records

Field

Value

crawl-delay

120

Back to top

Comments

================================================================
_ _ _ _
_ __ ___ | |__ ___ | |_ ___ | |___ _| |_
| '__/ _ \| '_ \ / _ \| __/ __| | __\ \/ / __|
| | | (_) | |_) | (_) | |_\__ \ _ | |_ > <| |_
|_| \___/|_.__/ \___/ \__|___/ (_) \__/_/\_\\__|
================================================================
f1a9{rO60t5.tXT_15_u5e1e55}
================================================================
========================================

Back to top

Warnings

8 invalid lines.

Back to top