kiteboarder-mag.com
robots.txt

Robots Exclusion Standard data for kiteboarder-mag.com

Resource Scan

Scan Details

Site Domain kiteboarder-mag.com
Base Domain kiteboarder-mag.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-12T12:00:25+00:00
Next Scan 2024-05-26T12:00:25+00:00

Last Successful Scan

Scanned2024-04-27T11:59:28+00:00
URL https://kiteboarder-mag.com/robots.txt
Redirect https://www.kiteboarder-mag.com/robots.txt
Redirect Domain www.kiteboarder-mag.com
Redirect Base kiteboarder-mag.com
Domain IPs 104.21.21.173, 172.67.199.167, 2606:4700:3031::6815:15ad, 2606:4700:3033::ac43:c7a7
Redirect IPs 104.21.21.173, 172.67.199.167, 2606:4700:3031::6815:15ad, 2606:4700:3033::ac43:c7a7
Response IP 172.67.199.167
Found Yes
Hash 9e2e9bbd07b42d91e9036f713b79000d5df9468afb10f27c971d72b7bb83f500
SimHash ad62fc148fc1

Groups

*

Rule Path
Disallow /admin/
Disallow /widget/
Disallow /xwidget/
Disallow *sort%3D*
Disallow *resultat-recherche*
Disallow *resultats-recherche*
Disallow /mentions-legales
Disallow /resultat-recherche-aile-kite
Disallow /resultats-recherche-planche-kite
Disallow /2015-ailes-accueil-tests
Disallow /maintenance
Disallow /flysurf
Disallow /chinook
Disallow /app-store
Disallow /partenaire-hotmer
Disallow /conseil-matos-test

nutch

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.kiteboarder-mag.com/sitemap.xml

Comments

  • User rules