/.well-known/

Log In Sign Up

pornolegende.com
robots.txt

Robots Exclusion Standard data for pornolegende.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	pornolegende.com
Base Domain	pornolegende.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Server returned a server error.
Last Scan	2024-09-12T04:34:33+00:00
Next Scan	2024-12-11T04:34:33+00:00

Last Successful Scan

Scanned	2024-01-23T19:06:05+00:00
URL	https://pornolegende.com/robots.txt
Redirect	https://www.pornolegende.net/robots.txt
Redirect Domain	www.pornolegende.net
Redirect Base	pornolegende.net
Domain IPs	104.21.16.130, 172.67.212.183, 2606:4700:3035::6815:1082, 2606:4700:3036::ac43:d4b7
Redirect IPs	104.21.51.35, 172.67.220.121, 2606:4700:3031::6815:3323, 2606:4700:3031::ac43:dc79
Response IP	172.67.220.121
Found	Yes
Hash	1fce2a930a3ce61b9579ce9beb707e94703d5b7e60f8346ae7ee6ce70f02e471
SimHash	f036e9684df2

Groups

*

Rule

Path

Allow

/

dotbot

Rule

Path

Disallow

/

giftghostbot

Rule

Path

Disallow

/

seznam

Rule

Path

Disallow

/

paperlibot

Rule

Path

Disallow

/

genieo

Rule

Path

Disallow

/

dataprovider/6.101

Rule

Path

Disallow

/

dataprovidersiteexplorer

Rule

Path

Disallow

/

dazoobot/1.0

Rule

Path

Disallow

/

diffbot

Rule

Path

Disallow

/

domainstatsbot/1.0

Rule

Path

Disallow

/

dotbot/1.1

Rule

Path

Disallow

/

dubaiindex

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

ecommercebot

Rule

Path

Disallow

/

expertsearchspider

Rule

Path

Disallow

/

feedbin

Rule

Path

Disallow

/

fetch/2.0a

Rule

Path

Disallow

/

ffbot/1.0

Rule

Path

Disallow

/

focusbot/1.1

Rule

Path

Disallow

/

huaweisymantecspider

Rule

Path

Disallow

/

huaweisymantecspider/1.0

Rule

Path

Disallow

/

jobdiggerspider

Rule

Path

Disallow

/

lemurwebcrawler

Rule

Path

Disallow

/

lipperheylinkexplorer

Rule

Path

Disallow

/

lssrocketcrawler/1.0

Rule

Path

Disallow

/

lyt.srv1.5

Rule

Path

Disallow

/

miadev/0.0.1

Rule

Path

Disallow

/

najdi.si/3.1

Rule

Path

Disallow

/

bountiibot

Rule

Path

Disallow

/

experibot_v1

Rule

Path

Disallow

/

bixocrawler

Rule

Path

Disallow

/

bixocrawler testcrawler

Rule

Path

Disallow

/

crawler4j

Rule

Path

Disallow

/

crowsnest/0.5

Rule

Path

Disallow

/

cukbot

Rule

Path

Disallow

/

dataprovider/6.92

Rule

Path

Disallow

/

dblbot/1.0

Rule

Path

Disallow

/

diffbot/0.1

Rule

Path

Disallow

/

digg deeper/v1

Rule

Path

Disallow

/

discobot/1.0

Rule

Path

Disallow

/

discobot/1.1

Rule

Path

Disallow

/

discobot/2.0

Rule

Path

Disallow

/

discoverybot/2.0

Rule

Path

Disallow

/

dlvr.it/1.0

Rule

Path

Disallow

/

domainstatsbot/1.0

Rule

Path

Disallow

/

drupact/0.7

Rule

Path

Disallow

/

ezooms/1.0

Rule

Path

Disallow

/

fastbot crawler beta 2.0

Rule

Path

Disallow

/

fastbot crawler beta 4.0

Rule

Path

Disallow

/

feedly social

Rule

Path

Disallow

/

feedly/1.0

Rule

Path

Disallow

/

feedlybot/1.0

Rule

Path

Disallow

/

feedspot

Rule

Path

Disallow

/

feedspotbot/1.0

Rule

Path

Disallow

/

clickagy intelligence bot v2

Rule

Path

Disallow

/

classbot

Rule

Path

Disallow

/

cispa vulnerability notification

Rule

Path

Disallow

/

cirrusexplorer/1.1

Rule

Path

Disallow

/

checksem/nutch-1.10

Rule

Path

Disallow

/

catchbot/5.0

Rule

Path

Disallow

/

catchbot/3.0

Rule

Path

Disallow

/

catchbot/2.0

Rule

Path

Disallow

/

catchbot/1.0

Rule

Path

Disallow

/

camontspider/1.0

Rule

Path

Disallow

/

buzzbot/1.0

Rule

Path

Disallow

/

buzzbot

Rule

Path

Disallow

/

businessseek.biz_spider

Rule

Path

Disallow

/

bubing

Rule

Path

Disallow

/

blexbot

Rule

Path

Disallow

/

fyberspider/1.3

Rule

Path

Disallow

/

findlinks/1.1.6-beta5

Rule

Path

Disallow

/

g2reader-bot/1.0

Rule

Path

Disallow

/

findlinks/1.1.6-beta6

Rule

Path

Disallow

/

findlinks/2.0

Rule

Path

Disallow

/

findlinks/2.0.1

Rule

Path

Disallow

/

findlinks/2.0.2

Rule

Path

Disallow

/

findlinks/2.0.4

Rule

Path

Disallow

/

findlinks/2.0.5

Rule

Path

Disallow

/

findlinks/2.0.9

Rule

Path

Disallow

/

findlinks/2.1

Rule

Path

Disallow

/

findlinks/2.1.5

Rule

Path

Disallow

/

findlinks/2.1.3

Rule

Path

Disallow

/

findlinks/2.2

Rule

Path

Disallow

/

findlinks/2.5

Rule

Path

Disallow

/

findlinks/2.6

Rule

Path

Disallow

/

ffbot/1.0

Rule

Path

Disallow

/

findlinks/1.0

Rule

Path

Disallow

/

findlinks/1.1.3-beta8

Rule

Path

Disallow

/

findlinks/1.1.3-beta9

Rule

Path

Disallow

/

findlinks/1.1.4-beta7

Rule

Path

Disallow

/

findlinks/1.1.6-beta1

Rule

Path

Disallow

/

findlinks/1.1.6-beta1 yacy

Rule

Path

Disallow

/

findlinks/1.1.6-beta2

Rule

Path

Disallow

/

findlinks/1.1.6-beta3

Rule

Path

Disallow

/

findlinks/1.1.6-beta4

Rule

Path

Disallow

/

bixo

Rule

Path

Disallow

/

bixolabs/1.0

Rule

Path

Disallow

/

crawlera/1.10.2

Rule

Path

Disallow

/

dataprovider site explorer

Rule

Path

Disallow

/

vagabondo

Rule

Path

Disallow

/

seokicks-robot

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

queryseekerspider

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

archive.org_bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

alexibot

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

surveybot

Rule

Path

Disallow

/

xenu's

Rule

Path

Disallow

/

xenu's link sleuth 1.1c

Rule

Path

Disallow

/

rogerbot

Rule

Path

Disallow

/

truliabot

Rule

Path

Disallow

/

feedjira

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

semrushbot-ba

Rule

Path

Disallow

/

semrushbot-si

Rule

Path

Disallow

/

semrushbot-swa

Rule

Path

Disallow

/

semrushbot-ct

Rule

Path

Disallow

/

semrushbot-bm

Rule

Path

Disallow

/

Back to top

Warnings

8 invalid lines.

Back to top