bologna.repubblica.it
robots.txt

Robots Exclusion Standard data for bologna.repubblica.it

Resource Scan

Scan Details

Site Domain bologna.repubblica.it
Base Domain repubblica.it
Scan Status Ok
Last Scan2024-05-11T15:59:04+00:00
Next Scan 2024-05-18T15:59:04+00:00

Last Scan

Scanned2024-05-11T15:59:04+00:00
URL https://bologna.repubblica.it/robots.txt
Domain IPs 13.224.163.14, 13.224.163.24, 13.224.163.31, 13.224.163.33
Response IP 52.84.229.5
Found Yes
Hash 34a28cd978ca08a631043eec4b158f06431d49a879212fa4153ca3e6a58e9e0c
SimHash 6a845929a033

Groups

*

Rule Path
Disallow /ristoranti/
Disallow /multimedia/
Disallow /dettaglio/
Disallow /dettaglio-news/
Disallow /blaize/datalayer

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://bologna.repubblica.it/sitemap-n.xml