Log In Sign Up

ccante1.free.fr
robots.txt

Robots Exclusion Standard data for ccante1.free.fr

Resource Scan

Scan Details

Site Domain ccante1.free.fr
Base Domain free.fr
Scan Status Ok
Last Scan2024-04-20T04:11:41+00:00
Next Scan 2024-05-20T04:11:41+00:00

Last Scan

Scanned2024-04-20T04:11:41+00:00
URL http://ccante1.free.fr/robots.txt
Domain IPs 212.27.63.133
Response IP 212.27.63.133
Found Yes
Hash 9a4ccfbd68a613b94003e6f1b5a80f345d014a8b27814ff55a243bcc9b192307
SimHash 2948d1905031

Groups

msnbot

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

googlebot

Rule Path
Disallow

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

fast-webcrawler

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

gigabotsitesearch

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

wisenutbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

searchpreview

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow

sbider

Rule Path
Disallow /

everbeecrawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

xget

Rule Path
Disallow /

psbot

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

phpdig

Rule Path
Disallow

*

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap http://ccante1.free.fr/googlemap1.xml
Back to top

Comments

  • /robots.txt file for http://ccante1.free.fr/
  • mail add_site2002@hotmail.com for constructive criticism
  • Tous les autres agents spiders
Back to top