marlinmag.com
robots.txt
Robots Exclusion Standard data for marlinmag.com
Resource Scan
Scan Details
| Site Domain | marlinmag.com |
| Base Domain | marlinmag.com |
| Scan Status | Ok |
| Last Scan | 2024-11-16T14:22:27+00:00 |
| Next Scan | 2024-11-23T14:22:27+00:00 |
Last Scan
| Scanned | 2024-11-16T14:22:27+00:00 |
| URL | https://marlinmag.com/robots.txt |
| Redirect | http://www.marlinmag.com/robots.txt |
| Redirect Domain | www.marlinmag.com |
| Redirect Base | marlinmag.com |
| Domain IPs | 141.193.213.20, 141.193.213.21 |
| Redirect IPs | 141.193.213.20, 141.193.213.21 |
| Response IP | 141.193.213.21 |
| Found | Yes |
| Hash | 85816b7991d45011a17f19f6ef72edaa3e7daa99679634e7aed7f934749b4580 |
| SimHash | 0f18df42f1e2 |
Groups
semrushbot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
splitsignalbot
semrushbot-coub
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /au/ |
| Disallow | /ca/ |
| Disallow | /fr/ |
| Disallow | /ca/ |
| Disallow | /fr/ |
| Disallow | /de/ |
| Disallow | /in/ |
| Disallow | /it/ |
| Disallow | /jp/ |
| Disallow | /mx/ |
| Disallow | /es/ |
| Disallow | /uk/ |
| Disallow | /comments/feed/ |
| Disallow | /trackback/ |
| Disallow | */comments$ |
| Disallow | */trackback |
| Disallow | */trackback$ |
| Disallow | /wp-comments |
| Disallow | /wp-trackback |
| Disallow | /cgi-bin/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-login.php |
| Disallow | /tags/* |
| Disallow | */page/* |
| Allow | */page/2/ |
| Allow | */page/3/ |
| Allow | */page/4/ |
| Allow | */page/5/ |
| Allow | */page/6/ |
| Allow | */page/7/ |
| Allow | */page/8/ |
| Allow | */page/9/ |
| Allow | */page/10/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.marlinmag.com/sitemap_index.xml |
Comments