mbspeedskating.com
robots.txt

Robots Exclusion Standard data for mbspeedskating.com

Resource Scan

Scan Details

Site Domain mbspeedskating.com
Base Domain mbspeedskating.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-05T16:56:42+00:00
Next Scan 2025-01-03T16:56:42+00:00

Last Successful Scan

Scanned2022-01-21T18:05:31+00:00
URL http://mbspeedskating.com/robots.txt
Redirect http://www.speedskatingmb.com/robots.txt
Redirect Domain www.speedskatingmb.com
Redirect Base speedskatingmb.com
Response IP 3.97.1.68
Found Yes
Hash 2449ff689d2891f7f2f34405cbd806ea0b4040aff6629694fffdbc18f5461d4c
SimHash ac1e19e87489

Groups

*

Rule Path
Disallow /*?SID=
Disallow /*%7B%7B
Disallow /*%7B%7B
Disallow /*?no_cache=
Disallow /*?nocache=
Disallow /tmp/
Disallow /vDev/
Disallow /vPreprod/
Disallow /vDemo/
Disallow /vBBQC/
Disallow /webmailAPIs/
Disallow /ctr/
Disallow /sponsors/
Disallow /adpics/
Disallow /vProd/iframeSession.php
Disallow /v5/
Disallow /v5dev/
Disallow /chrysophylax/
Disallow /ressources/files/
Disallow /fr/ms/reseaupublicationsports/
Disallow /en/ms/reseaupublicationsports/

Comments

  • Do not crawl links with session IDs
  • Do not crawl javascript links with {{token}}
  • Do not crawl links with ?no_cache