azloko.tv
robots.txt

Robots Exclusion Standard data for azloko.tv

Resource Scan

Scan Details

Site Domain azloko.tv
Base Domain azloko.tv
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-24T16:04:21+00:00
Next Scan 2024-08-22T16:04:21+00:00

Last Successful Scan

Scanned2023-01-27T01:57:08+00:00
URL https://azloko.tv/robots.txt
Redirect https://www.azloko.tv/robots.txt
Redirect Domain www.azloko.tv
Redirect Base azloko.tv
Domain IPs 104.21.37.205, 172.67.212.231, 2606:4700:3030::6815:25cd, 2606:4700:3032::ac43:d4e7
Redirect IPs 104.21.37.205, 172.67.212.231, 2606:4700:3030::6815:25cd, 2606:4700:3032::ac43:d4e7
Response IP 172.67.212.231
Found Yes
Hash 84459d8bf51990fce0786b694f7dc6ca37602cb4142e232c5ca7907ef02ddc51
SimHash cd2683638fb1

Groups

*

Rule Path
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /includes/
Disallow /lib/
Disallow /errors/
Disallow /magento/
Disallow /pkginfo/
Disallow /report/
Disallow /shell/
Disallow /stats/
Disallow /var/
Allow /media/catalog/product/
Allow /media/catalog/category/
Allow /media/wysiwyg/
Disallow /priceslider/
Disallow /checkout/
Disallow /ajaxcart/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /index.php/
Disallow /atendimento/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /control/
Disallow /contacts/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /sendfriend/
Disallow /tag/
Disallow /sales/
Disallow /reward/
Disallow /review/
Disallow /helpdesk/
Disallow /catalog/product/gallery/
Disallow /ajax/index/options/
Disallow /catalogsearch/
Disallow /catalogsearch/result/
Disallow /catalogsearch/result/index/
Disallow /searchautocomplete/ajax/suggest/*
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /*?price=*
Disallow /*%26price%3D*
Disallow /*?color=*
Disallow /*%26color%3D*
Disallow /*?limit=*
Disallow /*%26limit%3D*
Disallow /*%26p%3D*
Disallow /lojaconfiavel?p=*
Disallow /*?mode=*
Disallow /*%26mode%3D*
Disallow /*?order=*
Disallow /*%26order%3D*
Disallow /*?dir=*
Disallow /*%26dir%3D*
Disallow /*.php$
Disallow /*?SID=
Disallow /*?cat=*
Disallow /*?q=*

Comments

  • Crawlers Setup
  • Directories
  • Do not crawl checkout and user account pages
  • Paths (clean URLs)
  • Disallow: /home/
  • Files
  • Do not crawl sub category pages that are sorted or filtered.