Log In Sign Up

dentistionesti.it
robots.txt

Robots Exclusion Standard data for dentistionesti.it

Resource Scan

Scan Details

Site Domain dentistionesti.it
Base Domain dentistionesti.it
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-10-03T18:04:50+00:00
Next Scan 2024-10-10T18:04:50+00:00

Last Successful Scan

Scanned2024-09-25T17:42:53+00:00
URL https://dentistionesti.it/robots.txt
Domain IPs 185.224.137.21, 2a02:4780:8:1197:0:25a2:87fa:9
Response IP 185.224.137.21
Found Yes
Hash 85323bba9953bc165e040644a362f4f0347c9afd6e983c42a985e878c81bcb42
SimHash 1345d040c973

Groups

*

Rule Path
Disallow /*/user_registrazione_dentista-*

mozilla/5.0 (compatible; yandexbot/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

yandex

Rule Path
Disallow /

admantx

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

mozilla/5.0 (compatible;picmole/1.0 +http://www.picmole.com)

Rule Path
Disallow /

lexxebot/1.0

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

mozilla/5.0 (compatible; spbot/2.0; http://www.seoprofiler.com/bot/ )

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sosospider+(+http://help.soso.com/webspider.htm)

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sitebot/0.1

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

discobot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

sistrix

Rule Path
Disallow /

garlikcrawler/1.2 (http://garlik.com/, crawler@garlik.com)

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

addthis.com robot tech.support@clearspring.com

Rule Path
Disallow /

addthis.com

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

bender

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

linguee

Rule Path
Disallow /

integromedb

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /
Back to top