Log In Sign Up

tag.cl
robots.txt

Robots Exclusion Standard data for tag.cl

Resource Scan

Scan Details

Site Domain tag.cl
Base Domain tag.cl
Scan Status Ok
Last Scan2025-11-03T02:29:44+00:00
Next Scan 2025-11-10T02:29:44+00:00

Last Scan

Scanned2025-11-03T02:29:44+00:00
URL https://tag.cl/robots.txt
Domain IPs 107.180.113.152
Response IP 107.180.113.152
Found Yes
Hash d7a6482a29c9ad9df658d4f5ec08b57b934a7100298d0ad0e9837cb737d20b2e
SimHash 1825682b63f7

Groups

*

Rule Path
Disallow
Disallow /3rdparty/
Disallow /antispam.txt
Disallow /admin/
Disallow /admin/admin_index.php
Disallow /backup/
Disallow /cache/templates_c/
Disallow /cache/admin_c/
Disallow /install/
Disallow /internal/
Disallow /languages/
Disallow /libs/
Disallow /live/
Disallow /LICENSE.txt
Disallow /modules/
Disallow /plugins/
Disallow /readme.html
Disallow /search.php
Disallow /search/
Disallow /searchurl/
Disallow /spamlog.log
Disallow /tag/
Disallow /templates/
Disallow /user/profile/
Disallow /user/history/
Disallow /user/published/
Disallow /user/shaken/
Disallow /user/commented/
Disallow /user/voted/
Disallow /user/saved/
Disallow /recent/
Disallow /yesterday/
Disallow /today/
Disallow /week/
Disallow /month/
Disallow /year/
Disallow /alltime/
Disallow /upcoming/recent/
Disallow /upcoming/yesterday/
Disallow /upcoming/today/
Disallow /upcoming/week/
Disallow /upcoming/month/
Disallow /upcoming/year/
Disallow /upcoming/alltime/

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • 1) this filename (robots.txt) must stay lowercase
  • 2) this file must be in the servers root directory
  • ex: http://www.mydomain.com/pliggsubfolder/ -- you must move the robots.txt from
  • /pliggsubfolder/ to the root folder for http://www.mydomain.com/
  • you must then add your subfolder to each 'Disallow' below
  • ex: Disallow: /cache/ becomes Disallow: /pliggsubfolder/cache/
Back to top