Log In Sign Up

the-sports.org
robots.txt

Robots Exclusion Standard data for the-sports.org

Resource Scan

Scan Details

Site Domain the-sports.org
Base Domain the-sports.org
Scan Status Ok
Last Scan2024-06-06T20:49:44+00:00
Next Scan 2024-06-13T20:49:44+00:00

Last Scan

Scanned2024-06-06T20:49:44+00:00
URL https://the-sports.org/robots.txt
Redirect https://www.the-sports.org/robots.txt
Redirect Domain www.the-sports.org
Redirect Base the-sports.org
Domain IPs 104.21.71.187, 172.67.148.18, 2606:4700:3034::6815:47bb, 2606:4700:3037::ac43:9412
Redirect IPs 104.21.71.187, 172.67.148.18, 2606:4700:3034::6815:47bb, 2606:4700:3037::ac43:9412
Response IP 172.67.148.18
Found Yes
Hash 5f5e890d2ce3d505f217b0f73629ca6b24517cf859c18f6e56467bdb6c161049
SimHash 2a579c80c21f

Groups

mediapartners-google

Rule Path
Disallow

adsbot-google-mobile

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow /accueil.php
Disallow /index.php
Disallow /admin.php
Disallow /calendrier.php
Disallow /tmp/
Disallow /images/_bleu_elec
Disallow /images/boutons
Disallow /img/graph
Disallow /zz
Disallow /how-to-

linguee bot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

vegebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ezoicbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Disallow: /television_
Back to top