animalprive.com
robots.txt

Robots Exclusion Standard data for animalprive.com

Resource Scan

Scan Details

Site Domain animalprive.com
Base Domain animalprive.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-13T10:59:33+00:00
Next Scan 2026-03-13T10:59:33+00:00

Last Successful Scan

Scanned2025-04-25T09:20:39+00:00
URL https://animalprive.com/robots.txt
Redirect https://www.empruntemontoutou.com/robots.txt
Redirect Domain www.empruntemontoutou.com
Redirect Base empruntemontoutou.com
Domain IPs 217.70.184.55
Redirect IPs 104.26.14.32, 104.26.15.32, 172.67.74.148, 2606:4700:20::681a:e20, 2606:4700:20::681a:f20, 2606:4700:20::ac43:4a94
Response IP 104.26.15.32
Found Yes
Hash d0819c1b619738752ea1fd275ae8a397b00fefe29b5a9cc6ab49cdd2f129ffac
SimHash c0018a72c3b3

Groups

scrapy

Rule Path
Allow /

*

No rules defined. All paths allowed.

Comments

  • http://www.robotstxt.org/
  • https://support.google.com/webmasters/answer/6062608

Warnings

  • 1 invalid line.