Log In Sign Up

dreamfilmsw1.info
robots.txt

Robots Exclusion Standard data for dreamfilmsw1.info

Resource Scan

Scan Details

Site Domain dreamfilmsw1.info
Base Domain dreamfilmsw1.info
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-16T00:02:23+00:00
Next Scan 2024-11-23T00:02:23+00:00

Last Successful Scan

Scanned2024-10-16T00:01:11+00:00
URL https://dreamfilmsw1.info/robots.txt
Redirect https://dreamfilmsw.life/robots.txt
Redirect Domain dreamfilmsw.life
Redirect Base dreamfilmsw.life
Domain IPs 104.21.7.234, 172.67.156.135, 2606:4700:3036::6815:7ea, 2606:4700:3036::ac43:9c87
Redirect IPs 104.21.14.216, 172.67.160.152, 2606:4700:3033::ac43:a098, 2606:4700:3037::6815:ed8
Response IP 104.21.14.216
Found Yes
Hash 07bdd0675eb712f39693777af7ecf17e8d3d43c4014d82a8a1ed40be47bd34cf
SimHash 3a13ed7245ab

Groups

*

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

exabot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

majestic12

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

xenu

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

sitecheckerbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

advbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

tencenttraveler

Rule Path
Disallow /

zypp\ linkchecker

Rule Path
Disallow /

urlappendbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

salesforce

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /

oozbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

konqueror

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

kmlabsbot

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

houxoucrawler

Rule Path
Disallow /

htmlparser

Rule Path
Disallow /

haa(a)rtland

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

feedly

Rule Path
Disallow /

everyonesocialbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

dle_spider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

ca-crawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

browsershots

Rule Path
Disallow /

bubing

Rule Path
Disallow /

bubing

Rule Path
Disallow /

bubing

Rule Path
Disallow /

applebot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://dreamfilmsw.net/sitemap.index.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top