agenzie.generali.it
robots.txt

Robots Exclusion Standard data for agenzie.generali.it

Resource Scan

Scan Details

Site Domain agenzie.generali.it
Base Domain generali.it
Scan Status Ok
Last Scan2024-05-08T09:47:48+00:00
Next Scan 2024-06-07T09:47:48+00:00

Last Scan

Scanned2024-05-08T09:47:48+00:00
URL https://agenzie.generali.it/robots.txt
Redirect https://www.agenzie.generali.it/robots.txt
Redirect Domain www.agenzie.generali.it
Redirect Base generali.it
Domain IPs 45.223.167.17, 45.223.179.17
Redirect IPs 45.223.171.17
Response IP 45.223.171.17
Found Yes
Hash 50fa58da04aa149c47c69d72a423372301e01a6f29fd61edcfd872e71eaa31ed
SimHash 635d5a8048f1

Groups

*

Rule Path
Disallow

psbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

willybot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

germcrawler

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

xaldon_webspider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.agenzie.generali.it/sitemap.xml