Log In Sign Up

diversual.com
robots.txt

Robots Exclusion Standard data for diversual.com

Resource Scan

Scan Details

Site Domain diversual.com
Base Domain diversual.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-16T20:46:35+00:00
Next Scan 2024-09-30T20:46:35+00:00

Last Successful Scan

Scanned2024-08-09T20:45:29+00:00
URL https://diversual.com/robots.txt
Redirect https://www.diversual.com/robots.txt
Redirect Domain www.diversual.com
Redirect Base diversual.com
Domain IPs 3.165.102.12, 3.165.102.29, 3.165.102.61, 3.165.102.85
Redirect IPs 3.165.102.12, 3.165.102.29, 3.165.102.61, 3.165.102.85
Response IP 3.165.102.61
Found Yes
Hash ca02981364038c8c3d546b9a366b9789ee75adcb08f39641ad0efc8d17a5ce8e
SimHash 123de82a97b1

Groups

*

Rule Path
Disallow /*?tag=
Disallow /*?id_currency=
Disallow /*?search_query=
Disallow /*%26tag%3D
Disallow /*%26id_currency%3D
Disallow /*%26search_query%3D
Disallow /*%26back%3D
Disallow /*controller%3Daddresses
Disallow /*controller%3Daddress
Disallow /*controller%3Dauthentication
Disallow /*controller%3Dcart
Disallow /*controller%3Ddiscount
Disallow /*controller%3Dfooter
Disallow /*controller%3Dget-file
Disallow /*controller%3Dheader
Disallow /*controller%3Dhistory
Disallow /*controller%3Didentity
Disallow /*controller%3Dimages.inc
Disallow /*controller%3Dinit
Disallow /*controller%3Dmy-account
Disallow /*controller%3Dorder
Disallow /*controller%3Dorder-opc
Disallow /*controller%3Dorder-slip
Disallow /*controller%3Dorder-detail
Disallow /*controller%3Dorder-follow
Disallow /*controller%3Dorder-return
Disallow /*controller%3Dorder-confirmation
Disallow /*controller%3Dpagination
Disallow /*controller%3Dpassword
Disallow /*controller%3Dpdf-invoice
Disallow /*controller%3Dpdf-order-return
Disallow /*controller%3Dpdf-order-slip
Disallow /*controller%3Dproduct-sort
Disallow /*controller%3Dsearch
Disallow /*controller%3Dstatistics
Disallow /*controller%3Dattachment
Disallow /*controller%3Dguest-tracking
Disallow */classes/
Disallow */config/
Disallow */download/
Disallow */mails/
Disallow */translations/
Disallow */tools/
Disallow */ver-mas/instrucciones/pdf/
Disallow */s-1/
Disallow */s-2/
Disallow */s-3/
Disallow */s-4/
Disallow */s-5/
Disallow */s-6/
Disallow */s-7/
Disallow */s-8/
Disallow */s-9/
Disallow */doofinder/
Disallow */module/mailalerts/actions
Disallow */modules/blockcart
Disallow */modules/pm_advancedsearch4
Disallow */modules/paypal
Disallow */fernandez/factchecked.html
Disallow /*errordeblogposts
Disallow /*en/password-recovery
Disallow /*en/address
Disallow /*en/addresses
Disallow /*en/cart
Disallow /*en/discount
Disallow /*en/order-history
Disallow /*en/identity
Disallow /*en/order-follow
Disallow /*en/order-slip
Disallow /*en/order
Disallow /*en/search
Disallow /*en/guest-tracking
Disallow /*en/quick-order
Disallow /*fr/mot-de-passe-oublie
Disallow /*fr/adresse
Disallow /*fr/adresses
Disallow /*fr/panier
Disallow /*fr/bons-de-reduction
Disallow /*fr/historique-des-commandes
Disallow /*fr/identite
Disallow /*fr/details-de-la-commande
Disallow /*fr/avoirs
Disallow /*fr/commande
Disallow /*fr/recherche
Disallow /*fr/suivi-commande-invite
Disallow /*fr/commande-rapide
Disallow /*es/contrasena-olvidado
Disallow /*es/direccion
Disallow /*es/direcciones
Disallow /*es/descuento
Disallow /*es/historial-de-pedidos
Disallow /*es/identidad
Disallow /*es/devolucion-de-productos
Disallow /*es/vales
Disallow /*es/buscar
Disallow /*es/pedido-rapido
Disallow /*es/carrito-compra

nutch

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

url_spider_pro

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

web image collector

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

xenu's

Rule Path
Disallow /

zeus

Rule Path
Disallow /

zeus link scout

Rule Path
Disallow /

erocrawler

Rule Path
Disallow /

linkscan/8.1a unix

Rule Path
Disallow /

keyword density/0.9

Rule Path
Disallow /

webcopier v3.2a

Rule Path
Disallow /

webcapture 2.0

Rule Path
Disallow /

webcopier v.2.2

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

black hole

Rule Path
Disallow /

xenu\\\'s link sleuth 1.1c

Rule Path
Disallow /

xenu\\\'s

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.diversual.com/1_index_sitemap.xml
Back to top

Comments

  • ************************************
  • HAL9000
  • -------
  • DIVERSUAL
  • ************************************
  • PAGINAS PRIVADAS
  • DIRECTORIOS
  • Filtros de busqueda
  • Disalloow: */en/
  • Disalloow: */fr/
  • BLOG Se quitan idiomas <> ES, categorias y enlaces no amigables
  • Files
  • SITEMAP
  • SPECIFIC BOTS
  • Majestic solo cada 10 segundos
Back to top