dizionari.repubblica.it
robots.txt

Robots Exclusion Standard data for dizionari.repubblica.it

Resource Scan

Scan Details

Site Domain dizionari.repubblica.it
Base Domain repubblica.it
Scan Status Ok
Last Scan2024-05-14T04:43:58+00:00
Next Scan 2024-05-21T04:43:58+00:00

Last Scan

Scanned2024-05-14T04:43:58+00:00
URL https://dizionari.repubblica.it/robots.txt
Domain IPs 108.138.246.109, 108.138.246.2, 108.138.246.7, 108.138.246.75
Response IP 18.165.171.83
Found Yes
Hash 6828da91f75280635abc0b6ea0c7b8b427cae18df2932ca848001dd7e726287e
SimHash 28384b68a133

Groups

*

Rule Path
Disallow /cgi-bin/

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /