comlandi.fr
robots.txt

Robots Exclusion Standard data for comlandi.fr

Resource Scan

Scan Details

Site Domain comlandi.fr
Base Domain comlandi.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-03T16:01:23+00:00
Next Scan 2024-11-01T16:01:23+00:00

Last Successful Scan

Scanned2023-06-18T15:58:55+00:00
URL https://www.comlandi.fr/robots.txt
Domain IPs 96.17.96.10, 96.17.96.5
Response IP 23.33.126.7
Found Yes
Hash a8bb454da15b41bb262cc8bb9933fcc13a98c92634008b1f9f3518ea4c1f71fd
SimHash 031ac09156c1

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Comments

  • Intervalo de 10 segundos en bots que no sean de Google, ya que este no considera esta opcion