crave.ca
robots.txt
Robots Exclusion Standard data for crave.ca
Resource Scan
Scan Details
| Site Domain | crave.ca |
| Base Domain | crave.ca |
| Scan Status | Ok |
| Last Scan | 2024-11-14T00:58:47+00:00 |
| Next Scan | 2024-11-21T00:58:47+00:00 |
Last Scan
| Scanned | 2024-11-14T00:58:47+00:00 |
| URL | https://crave.ca/robots.txt |
| Redirect | https://www.crave.ca/robots.txt |
| Redirect Domain | www.crave.ca |
| Redirect Base | crave.ca |
| Domain IPs | 23.15.145.114, 2600:1413:1:397::3f3b, 2600:1413:1:3a1::3f3b |
| Redirect IPs | 23.50.82.20, 2600:1413:1:989::3f3b |
| Response IP | 23.50.82.20 |
| Found | Yes |
| Hash | bcdfd70312a0f468050b76163eec47a1c25e6a4b85049a066e42590627ee5b8a |
| SimHash | 1d0c1da47790 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*?ua=* |
| Disallow | /debug |
| Disallow | /en/profiles? |
| Disallow | /fr/profiles? |
| Disallow | /en/profiles/* |
| Disallow | /fr/profiles/* |
| Disallow | /en/search/ |
| Disallow | /fr/search/ |
| Disallow | */search/ |
| Disallow | /hotdtraining |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.crave.ca/sitemap.xml |
| sitemap | https://www.crave.ca/video-sitemap/video-sitemap.xml |