Log In Sign Up

creditospersonales.us
robots.txt

Robots Exclusion Standard data for creditospersonales.us

Resource Scan

Scan Details

Site Domain creditospersonales.us
Base Domain creditospersonales.us
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-09-10T19:50:03+00:00
Next Scan 2024-12-09T19:50:03+00:00

Last Successful Scan

Scanned2022-08-05T23:02:58+00:00
URL https://creditospersonales.us/robots.txt
Response IP 185.2.151.37
Found Yes
Hash a675ac3d7cd9927d187bc1694975c57c94fa09a156df0230bd1ee4579cc5bd5e
SimHash 887c52000413

Groups

*

Rule Path
Disallow /?s=
Disallow /search
Allow /feed/$
Disallow /feed
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mediapartners-google

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap http://losbancos.us/sitemap.xml
Back to top

Comments

  • Sitemap permitido, busquedas no.
  • Permitimos el feed general para Google Blogsearch.
  • Impedimos que permalink/feed/ sea indexado ya que el
  • feed con los comentarios suele posicionarse en lugar de
  • la entrada y desorienta a los usuarios.
  • Lo mismo con URLs terminadas en /trackback/ que solo
  • sirven como Trackback URI (y son contenido duplicado).
  • A partir de aqui­ es opcional pero recomendado.
  • Lista de bots que suelen respetar el robots.txt pero rara
  • vez hacen un buen uso del sitio y abusan bastante...
  • Anyadir al gusto del consumidor...
Back to top

Warnings

  • 4 invalid lines.
Back to top