agriturismo-inn.it
robots.txt

Robots Exclusion Standard data for agriturismo-inn.it

Resource Scan

Scan Details

Site Domain agriturismo-inn.it
Base Domain agriturismo-inn.it
Scan Status Ok
Last Scan2024-05-29T20:09:39+00:00
Next Scan 2024-06-28T20:09:39+00:00

Last Scan

Scanned2024-05-29T20:09:39+00:00
URL https://agriturismo-inn.it/robots.txt
Redirect https://www.agriturismo-inn.it/robots.txt
Redirect Domain www.agriturismo-inn.it
Redirect Base agriturismo-inn.it
Domain IPs 104.21.26.197, 172.67.168.129, 2606:4700:3032::ac43:a881, 2606:4700:3035::6815:1ac5
Redirect IPs 104.21.26.197, 172.67.168.129, 2606:4700:3032::ac43:a881, 2606:4700:3035::6815:1ac5
Response IP 172.67.168.129
Found Yes
Hash 5ade4b3e9ed90af0e766dfa9ef95561711d62feb77d825a2dcb1375acb9e3d88
SimHash 566541d0cfc8

Groups

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

linespider

Rule Path
Disallow /

infotigerbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

amazon-kendra-web-crawler-*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

*

Rule Path
Disallow /pub/
Disallow /inc/
Disallow /action/
Disallow /autosearch/
Disallow /policies/