refugeforums.com
robots.txt

Robots Exclusion Standard data for refugeforums.com

Resource Scan

Scan Details

Site Domain refugeforums.com
Base Domain refugeforums.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-15T03:48:17+00:00
Next Scan 2025-01-13T03:48:17+00:00

Last Successful Scan

Scanned2023-09-21T15:21:21+00:00
URL https://refugeforums.com/robots.txt
Domain IPs 104.21.24.133, 172.67.218.241, 2606:4700:3032::6815:1885, 2606:4700:3036::ac43:daf1
Response IP 172.67.218.241
Found Yes
Hash b1782b4542c0e3667eb74957f6c2a655d11f8c5026eb266ff73ba0e7358180c5
SimHash 4550dc724735

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2