criterionforum.org
robots.txt

Robots Exclusion Standard data for criterionforum.org

Resource Scan

Scan Details

Site Domain criterionforum.org
Base Domain criterionforum.org
Scan Status Ok
Last Scan2024-11-13T17:14:59+00:00
Next Scan 2024-11-20T17:14:59+00:00

Last Scan

Scanned2024-11-13T17:14:59+00:00
URL https://criterionforum.org/robots.txt
Domain IPs 2406:da18:9d0:143f:2124:4e9c:36a9:d9de, 52.221.42.138
Response IP 52.221.42.138
Found Yes
Hash 512bb6507089a03c9289a1b2f0d6dbc93398a18515f98a228787b8271b4505c6
SimHash 0c3c4cc0ebd6

Groups

*

Rule Path
Disallow /cron.php
Disallow /faq.php
Disallow /mcp.php
Disallow /memberlist.php
Disallow /posting.php
Disallow /report.php
Disallow /search.php
Disallow /ucp.php

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

Comments

  • Block wildcard bots from common
  • Block bots that respect robots.txt