tournoistubalde.com
robots.txt

Robots Exclusion Standard data for tournoistubalde.com

Resource Scan

Scan Details

Site Domain tournoistubalde.com
Base Domain tournoistubalde.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-07-23T08:27:44+00:00
Next Scan 2024-10-21T08:27:44+00:00

Last Successful Scan

Scanned2023-09-28T05:11:24+00:00
URL https://tournoistubalde.com/robots.txt
Redirect http://www.tournoistubalde.com/robots.txt
Redirect Domain www.tournoistubalde.com
Redirect Base tournoistubalde.com
Domain IPs 3.98.105.191
Redirect IPs 3.97.1.68, 3.98.81.84, 99.79.174.176
Response IP 3.97.1.68
Found Yes
Hash fabbc5ce0838b41ad4694a1a9816cb40e23c4a4d3ce94514b0e78110cd1d3f01
SimHash ac1e9dfa760b

Groups

*

Rule Path
Disallow /*%7B%7B
Disallow /*%7B%7B
Disallow /*?SID=
Disallow /*?no_cache=
Disallow /*?nocache=
Disallow /tmp/
Disallow /vDev/
Disallow /vPreprod/
Disallow /webmailAPIs/
Disallow /ctr/
Disallow /sponsors/
Disallow /adpics/
Disallow /vProd/iframeSession.php
Disallow /v5/
Disallow /v5dev/
Disallow /chrysophylax/
Disallow /ressources/files/
Disallow /fr/ms/reseaupublicationsports/
Disallow /en/ms/reseaupublicationsports/

qwantify

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

brightbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

Comments

  • Do not crawl javascript links with {{token}}
  • Do not crawl links with ?no_cache
  • Disallow some bots
  • Disallow Bad bots

Warnings

  • 2 invalid lines.