conscienceuniverselle.fr
robots.txt

Robots Exclusion Standard data for conscienceuniverselle.fr

Resource Scan

Scan Details

Site Domain conscienceuniverselle.fr
Base Domain conscienceuniverselle.fr
Scan Status Ok
Last Scan2024-09-21T08:09:29+00:00
Next Scan 2024-09-28T08:09:29+00:00

Last Scan

Scanned2024-09-21T08:09:29+00:00
URL http://conscienceuniverselle.fr/robots.txt
Redirect http://www.conscienceuniverselle.fr/robots.txt
Redirect Domain www.conscienceuniverselle.fr
Redirect Base conscienceuniverselle.fr
Domain IPs 213.186.33.5
Redirect IPs 213.186.33.2
Response IP 213.186.33.2
Found Yes
Hash 15edcd199addedaa3f4a8c31ee74ce302e2d65c932457d9b4f919662a5f98c57
SimHash 8c594142c7f3

Groups

*

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

bdbrandprotect

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /