francobordo.com
robots.txt

Robots Exclusion Standard data for francobordo.com

Resource Scan

Scan Details

Site Domain francobordo.com
Base Domain francobordo.com
Scan Status Ok
Last Scan2024-06-11T07:59:48+00:00
Next Scan 2024-07-11T07:59:48+00:00

Last Scan

Scanned2024-06-11T07:59:48+00:00
URL https://francobordo.com/robots.txt
Redirect https://www.francobordo.com/robots.txt
Redirect Domain www.francobordo.com
Redirect Base francobordo.com
Domain IPs 20.71.1.14
Redirect IPs 104.26.10.119, 104.26.11.119, 172.67.69.160, 2606:4700:20::681a:a77, 2606:4700:20::681a:b77, 2606:4700:20::ac43:45a0
Response IP 172.67.69.160
Found Yes
Hash c00b262f05772cdda42d9abea3fbc004ad33e808a8623db579c4b881c0cdc168
SimHash a15ac9004ed3

Groups

*

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /emails/
Disallow /temp/
Disallow /ext/
Disallow /chat/
Disallow /lists/
Disallow /feeds/
Disallow /fm-feeds/
Disallow /googlesitemap/
Disallow /html2pdf/
Disallow /upload/
Disallow /feed/
Disallow /404.php
Disallow /popup_image.php
Disallow /popup_search_help.php
Disallow /account.php
Disallow /account_edit.php
Disallow /account_history.php
Disallow /account_history_info.php
Disallow /account_password.php
Disallow /add_checkout_success.php
Disallow /address_book.php
Disallow /address_book_process.php
Disallow /advanced_search.php
Disallow /advanced_search_result.php
Disallow /advanced_search_result.php*
Disallow /comparador.php
Disallow /checkout_confirmation.php
Disallow /checkout_payment.php
Disallow /checkout_payment_address.php
Disallow /checkout_process.php
Disallow /checkout_shipping.php
Disallow /checkout_shipping_address.php
Disallow /checkout_success.php
Disallow /contact_bean.php
Disallow /cookie_usage.php
Disallow /create_account.php
Disallow /create_account_success.php
Disallow /login.php
Disallow /password_forgotten.php
Disallow /popup_image.php
Disallow /product_thumb.php
Disallow /shopping_cart.php
Disallow /product_reviews_write.php
Disallow /advanced_search.php*
Disallow /index.php*
Disallow /*action%3D*

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

linguee

Rule Path
Disallow /

archive.org

Rule Path
Disallow /

kalooga

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sbider

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap http://www.francobordo.com/sitemap.xml
sitemap https://www.francobordo.com/sitemapproducts.xml
sitemap https://www.francobordo.com/sitemapcategories.xml
sitemap https://www.francobordo.com/sitemapimages.xml
sitemap https://www.francobordo.com/sitemapmanufacturers.xml

Comments

  • Lista de bots que suelen respetar el robots.txt pero rara
  • vez hacen un buen uso del sitio y abusan bastante
  • AƱadir al gusto del consumidor
  • Slurp (Yahoo!), Noxtrum y el bot de MSN a veces tienen
  • idas de pinza, toca decirles que reduzcan la marcha.
  • El valor es en segundos y podeis dejarlo bajo e ir
  • subiendo hasta el punto optimo.