dive-the-world.com
robots.txt
Robots Exclusion Standard data for dive-the-world.com
Resource Scan
Scan Details
| Site Domain | dive-the-world.com |
| Base Domain | dive-the-world.com |
| Scan Status | Ok |
| Last Scan | 2025-11-23T22:26:51+00:00 |
| Next Scan | 2025-12-07T22:26:51+00:00 |
Last Scan
| Scanned | 2025-11-23T22:26:51+00:00 |
| URL | https://dive-the-world.com/robots.txt |
| Redirect | https://www.dive-the-world.com/robots.txt |
| Redirect Domain | www.dive-the-world.com |
| Redirect Base | dive-the-world.com |
| Domain IPs | 104.21.25.193, 172.67.134.127, 2606:4700:3031::ac43:867f, 2606:4700:3033::6815:19c1 |
| Redirect IPs | 104.21.25.193, 172.67.134.127, 2606:4700:3031::ac43:867f, 2606:4700:3033::6815:19c1 |
| Response IP | 172.67.134.127 |
| Found | Yes |
| Hash | 6905531c30d5d60b0988a8c023646e6e7d9c3ff3ff01a857e4f68ab71a048850 |
| SimHash | 71279801c992 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /test/ |
| Disallow | /sales/ |
| Disallow | /Connections/ |
| Disallow | /ads/ |
| Disallow | /admin/ |
| Disallow | /carousel/ |
| Disallow | /class/ |
| Disallow | /demo/ |
| Disallow | /enqs/ |
| Disallow | /include/ |
| Disallow | /partners/ |
| Disallow | /resort/ |
| Disallow | /phpmailer/ |
| Disallow | /schema/ |
| Disallow | /scripts/ |
| Disallow | /de/ |
| Disallow | /cn/ |
| Disallow | /m/ |
| Disallow | /mobile/ |
| Disallow | /search_results_lvb* |
| Disallow | /search_iframe* |
| Disallow | /files/ |
| Disallow | /dev/ |
| Disallow | /dev2/ |
| Disallow | /js1000/ |
| Disallow | /sass/ |
| Disallow | /js/ |
| Disallow | /.cache/ |
| Disallow | *?*arrdate= |
| Disallow | *?*currency= |
| Disallow | *?*ref= |
| Disallow | *?*y=20 |
| Disallow | *?*loc_id= |
| Disallow | *?*page_id= |
| Disallow | *?*location= |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.dive-the-world.com/sitemap.xml |
Comments