caravanersforum.com
robots.txt
Robots Exclusion Standard data for caravanersforum.com
Resource Scan
Scan Details
| Site Domain | caravanersforum.com |
| Base Domain | caravanersforum.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-07-07T10:09:07+00:00 |
| Next Scan | 2025-10-05T10:09:07+00:00 |
Last Successful Scan
| Scanned | 2024-05-21T09:54:55+00:00 |
| URL | https://caravanersforum.com/robots.txt |
| Redirect | https://www.caravanersforum.com/robots.txt |
| Redirect Domain | www.caravanersforum.com |
| Redirect Base | caravanersforum.com |
| Domain IPs | 43.250.142.144 |
| Redirect IPs | 43.250.142.144 |
| Response IP | 43.250.142.144 |
| Found | Yes |
| Hash | 160ffa28e9f22b37a14b2cd5b37ea5828fd91cb53a51799963aad4c6fe696ad9 |
| SimHash | 2ad65c744444 |
Groups
googlebot
google
bingbot
| Rule | Path |
|---|---|
| Disallow | /bin/ |
| Disallow | /cgi-bin/ |
| Disallow | /tmp/ |
| Disallow | /adm/ |
| Disallow | /assets/ |
| Disallow | /config/ |
| Disallow | /cache/ |
| Disallow | /docs/ |
| Disallow | /download/ |
| Disallow | /ext/ |
| Disallow | /files/ |
| Disallow | /images/ |
| Disallow | /includes/ |
| Disallow | /language/ |
| Disallow | /store/ |
| Disallow | /styles/ |
| Disallow | /vendor/ |
| Disallow | /mobiquo/ |
| Disallow | /phpbb/ |
| Disallow | /forumshop/ |
| Disallow | /config.php |
| Disallow | /cron.php |
| Disallow | /campslinks.php |
| Disallow | /campscount.php |
| Disallow | /campscounts.txt |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
*
| Rule | Path |
|---|---|
| Disallow | / |