azloko.com
robots.txt

Robots Exclusion Standard data for azloko.com

Resource Scan

Scan Details

Site Domain azloko.com
Base Domain azloko.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-23T13:08:39+00:00
Next Scan 2024-08-21T13:08:39+00:00

Last Successful Scan

Scanned2023-01-25T16:14:25+00:00
URL https://azloko.com/robots.txt
Redirect https://www.azloko.tv/robots.txt
Redirect Domain www.azloko.tv
Redirect Base azloko.tv
Domain IPs 104.21.1.4, 172.67.186.113, 2606:4700:3030::ac43:ba71, 2606:4700:3037::6815:104
Redirect IPs 104.21.37.205, 172.67.212.231, 2606:4700:3030::6815:25cd, 2606:4700:3032::ac43:d4e7
Response IP 104.21.37.205
Found Yes
Hash 84459d8bf51990fce0786b694f7dc6ca37602cb4142e232c5ca7907ef02ddc51
SimHash cd2683638fb1

Groups

*

Rule Path
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /includes/
Disallow /lib/
Disallow /errors/
Disallow /magento/
Disallow /pkginfo/
Disallow /report/
Disallow /shell/
Disallow /stats/
Disallow /var/
Allow /media/catalog/product/
Allow /media/catalog/category/
Allow /media/wysiwyg/
Disallow /priceslider/
Disallow /checkout/
Disallow /ajaxcart/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /index.php/
Disallow /atendimento/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /control/
Disallow /contacts/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /sendfriend/
Disallow /tag/
Disallow /sales/
Disallow /reward/
Disallow /review/
Disallow /helpdesk/
Disallow /catalog/product/gallery/
Disallow /ajax/index/options/
Disallow /catalogsearch/
Disallow /catalogsearch/result/
Disallow /catalogsearch/result/index/
Disallow /searchautocomplete/ajax/suggest/*
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /*?price=*
Disallow /*%26price%3D*
Disallow /*?color=*
Disallow /*%26color%3D*
Disallow /*?limit=*
Disallow /*%26limit%3D*
Disallow /*%26p%3D*
Disallow /lojaconfiavel?p=*
Disallow /*?mode=*
Disallow /*%26mode%3D*
Disallow /*?order=*
Disallow /*%26order%3D*
Disallow /*?dir=*
Disallow /*%26dir%3D*
Disallow /*.php$
Disallow /*?SID=
Disallow /*?cat=*
Disallow /*?q=*

Comments

  • Crawlers Setup
  • Directories
  • Do not crawl checkout and user account pages
  • Paths (clean URLs)
  • Disallow: /home/
  • Files
  • Do not crawl sub category pages that are sorted or filtered.