arabian-desert.com
robots.txt
Robots Exclusion Standard data for arabian-desert.com
Resource Scan
Scan Details
| Site Domain | arabian-desert.com |
| Base Domain | arabian-desert.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2026-02-02T14:51:58+00:00 |
| Next Scan | 2026-03-04T14:51:58+00:00 |
Last Successful Scan
| Scanned | 2025-12-12T13:57:14+00:00 |
| URL | https://arabian-desert.com/robots.txt |
| Domain IPs | 191.101.228.15, 2a02:4780:16:ef85:e8a3:e40b:9d2a:607d, 2a02:4780:38:e1a8:c631:2454:6db5:4005, 77.37.48.8 |
| Response IP | 77.37.48.204 |
| Found | Yes |
| Hash | 7a1934bbed761c5214dce8c74d65629c7f5b805cf0620c2c65537f6e0efaa8b4 |
| SimHash | 6d00d214a791 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /admin/ |
| Disallow | /private/ |
| Disallow | /*.json$ |
| Disallow | /*.xml$ |
| Allow | /*.jpg$ |
| Allow | /*.jpeg$ |
| Allow | /*.gif$ |
| Allow | /*.png$ |
| Allow | /*.webp$ |
| Allow | /*.svg$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://arabian-desert.com/sitemap.xml |
Comments