gitlab.mirsantov.com
robots.txt

Robots Exclusion Standard data for gitlab.mirsantov.com

Resource Scan

Scan Details

Site Domain gitlab.mirsantov.com
Base Domain mirsantov.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2024-10-06T02:21:45+00:00
Next Scan 2024-10-07T02:21:45+00:00

Last Successful Scan

Scanned2024-10-05T02:21:30+00:00
URL http://gitlab.mirsantov.com/robots.txt
Domain IPs 109.107.189.98
Response IP 109.107.189.98
Found Yes
Hash 8923abb0cf3cb4361a4c4c62e26f6f8d51ad5395e9db5bccbd466e66b50cbf30
SimHash 4854cb006773

Groups

*

Rule Path
Disallow /casino

*

Rule Path
Allow /

Comments

  • Allow all crawlers