Log In Sign Up

ilfattoquotidiano.it.webstatdata.com
robots.txt

Robots Exclusion Standard data for ilfattoquotidiano.it.webstatdata.com

Resource Scan

Scan Details

Site Domain ilfattoquotidiano.it.webstatdata.com
Base Domain webstatdata.com
Scan Status Ok
Last Scan2024-04-23T13:28:36+00:00
Next Scan 2024-05-23T13:28:36+00:00

Last Scan

Scanned2024-04-23T13:28:36+00:00
URL https://ilfattoquotidiano.it.webstatdata.com/robots.txt
Domain IPs 104.21.52.75, 172.67.196.178, 2606:4700:3031::6815:344b, 2606:4700:3031::ac43:c4b2
Response IP 104.21.52.75
Found Yes
Hash 9d6d8656d7927e03490952a9870996e789544166e7b1541784a15d08d35166f7
SimHash fa1cd841f62a

Groups

*

Rule Path
Disallow /widget/
Disallow /task.php
Disallow /widget.php

hypestat

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.webstatdata.com/sitemap/
Back to top

Comments

  • BEGIN XML Sitemap
  • END XML Sitemap
Back to top