bolognatoday.it
robots.txt

Robots Exclusion Standard data for bolognatoday.it

Resource Scan

Scan Details

Site Domain bolognatoday.it
Base Domain bolognatoday.it
Scan Status Ok
Last Scan2024-06-11T22:43:58+00:00
Next Scan 2024-06-18T22:43:58+00:00

Last Scan

Scanned2024-06-11T22:43:58+00:00
URL https://bolognatoday.it/robots.txt
Redirect https://www.bolognatoday.it/robots.txt
Redirect Domain www.bolognatoday.it
Redirect Base bolognatoday.it
Domain IPs 109.168.105.232, 109.168.105.233, 195.231.59.141, 46.254.34.118, 52.144.69.196, 52.144.69.197, 95.110.224.15
Redirect IPs 109.168.105.232, 109.168.105.233, 195.231.59.141, 46.254.34.118, 52.144.69.196, 52.144.69.197, 95.110.224.15
Response IP 52.144.69.197
Found Yes
Hash 669fff374b8f28f28bdcd6352725c1e89d4e82fdcaed400096a848f14353d018
SimHash 0bcdca2047d0

Groups

*

Rule Path
Allow /
Disallow /~vda/
Disallow /~shared/do/
Disallow /~shared/cgi-bin/
Disallow /~do/
Disallow /~cgi-bin/
Disallow /do/
Disallow /cgi-bin/
Disallow /~test/
Disallow /~api/
Disallow /~ajax/
Disallow /~otp/
Disallow /~pixel/
Disallow /~empty/
Disallow /captcha/
Disallow /form/
Disallow /signup/
Disallow /commento/
Disallow /user/login/
Disallow /user/logout/
Disallow /user/sso/
Disallow /user/oauth/
Disallow /user/activate/
Disallow /user/reset/
Disallow /user/delete/
Disallow /user/unsubscribe/
Disallow /user/contents/
Disallow /user/news/
Disallow /user/relation/
Disallow /user/subscription/
Disallow /user/edit/
Disallow /user/self/
Disallow /~shared/styles/
Disallow /styles/
Disallow /~shared/scripts/
Disallow /scripts/
Disallow /medias/
Disallow /uploads/
Disallow /sf/
Allow /~shared/do/api/google/
Allow /~shared/do/api/google-newsstand/
Allow /~shared/do/api/amazon/
Allow /~shared/do/api/facebook/
Allow /~shared/do/api/samsung/

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.bolognatoday.it/sitemaps/sitemap.xml
sitemap https://www.bolognatoday.it/sitemaps/sitemap_news.xml

Warnings

  • `host` is not a known field.