magicseaweed.com
robots.txt
Robots Exclusion Standard data for magicseaweed.com
Resource Scan
Scan Details
| Site Domain | magicseaweed.com |
| Base Domain | magicseaweed.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-10-12T00:43:30+00:00 |
| Next Scan | 2025-01-10T00:43:30+00:00 |
Last Successful Scan
| Scanned | 2023-06-20T23:59:20+00:00 |
| URL | https://magicseaweed.com/robots.txt |
| Domain IPs | 104.18.28.248, 104.18.29.248, 2606:4700::6812:1cf8, 2606:4700::6812:1df8 |
| Response IP | 104.18.29.248 |
| Found | Yes |
| Hash | 8f7aef063a7f6ac3100cbd3aa50452af4515c9cb93a0fdb704cd434f0d837db2 |
| SimHash | a6255e516fc8 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /out.php |
| Disallow | /account/ |
| Disallow | /surfmap/ |
| Disallow | /dirOut.php |
| Disallow | /photoLab/uploadPhoto.php |
| Disallow | /login.php |
| Disallow | /photoLab/tagPhoto.php |
| Disallow | /spot-forecast-detail.php |
| Disallow | /spot-daily-history.php |
| Disallow | /comment/post.php |
| Disallow | /comment/rate.php |
| Disallow | /photoLab/tag-history.php |
| Disallow | /wap/ |
| Disallow | /*-Surf-Report/*/Multi-Model-Forecast/?* |
| Disallow | /*-Surf-Report/*/*?enableForecastBeta |
| Disallow | /iframe/ |
| Disallow | /developers/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /*-Surf-Report/Tide/?start=* |
| Disallow | /out.php |
| Disallow | /account/ |
| Disallow | /surfmap/ |
| Disallow | /dirOut.php |
| Disallow | /photoLab/uploadPhoto.php |
| Disallow | /login.php |
| Disallow | /photoLab/tagPhoto.php |
| Disallow | /spot-forecast-detail.php |
| Disallow | /spot-daily-history.php |
| Disallow | /comment/post.php |
| Disallow | /comment/rate.php |
| Disallow | /photoLab/tag-history.php |
| Disallow | /wap/ |
| Disallow | /*-Surf-Report/*/Multi-Model-Forecast/?* |
| Disallow | /*-Surf-Report/*/*?enableForecastBeta |
| Disallow | /iframe/ |
| Disallow | /developers/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 50 |