avvenire.it
robots.txt

Robots Exclusion Standard data for avvenire.it

Resource Scan

Scan Details

Site Domain avvenire.it
Base Domain avvenire.it
Scan Status Ok
Last Scan2024-09-16T04:18:18+00:00
Next Scan 2024-09-23T04:18:18+00:00

Last Scan

Scanned2024-09-16T04:18:18+00:00
URL https://avvenire.it/robots.txt
Redirect https://www.avvenire.it/robots.txt
Redirect Domain www.avvenire.it
Redirect Base avvenire.it
Domain IPs 52.174.92.153
Redirect IPs 52.174.92.153
Response IP 52.174.92.153
Found Yes
Hash 1e78524c38e4b53372388f29910e096f830fc16b7ad75fd6f81719b03e0dd6ff
SimHash f347e994e145

Groups

*

Rule Path
Disallow /test
Disallow /cassetto
Disallow /shared
Disallow /info/seminaristi
Disallow /photogallery
Disallow /chiesa-e-societa
Disallow /comunicazione
Disallow /Culture
Disallow /percorsi-di-teologia
Disallow /vita-ecclesiale
Disallow /newsshowcase
Disallow /newsshowcasehome

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

googlebot/2.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

googlebot-image

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Warnings

  • 2 invalid lines.