agriturismo-vacanza.it
robots.txt

Robots Exclusion Standard data for agriturismo-vacanza.it

Resource Scan

Scan Details

Site Domain agriturismo-vacanza.it
Base Domain agriturismo-vacanza.it
Scan Status Ok
Last Scan2024-05-28T16:16:54+00:00
Next Scan 2024-06-27T16:16:54+00:00

Last Scan

Scanned2024-05-28T16:16:54+00:00
URL https://agriturismo-vacanza.it/robots.txt
Redirect https://www.agriturismo-vacanza.it/robots.txt
Redirect Domain www.agriturismo-vacanza.it
Redirect Base agriturismo-vacanza.it
Domain IPs 104.21.62.24, 172.67.218.163, 2606:4700:3032::6815:3e18, 2606:4700:3035::ac43:daa3
Redirect IPs 104.21.62.24, 172.67.218.163, 2606:4700:3032::6815:3e18, 2606:4700:3035::ac43:daa3
Response IP 104.21.62.24
Found Yes
Hash ddc9d4a2e192b9b2792b49fac875e21c59f0101febda4c0292efb3fcebd3d17e
SimHash 566541d0cfc8

Groups

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

linespider

Rule Path
Disallow /

infotigerbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

amazon-kendra-web-crawler-*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

*

Rule Path
Disallow /pub/
Disallow /inc/
Disallow /action/
Disallow /autosearch/
Disallow /policies/