Log In Sign Up

gipsytv.eu
robots.txt

Robots Exclusion Standard data for gipsytv.eu

Resource Scan

Scan Details

Site Domain gipsytv.eu
Base Domain gipsytv.eu
Scan Status Ok
Last Scan2024-09-29T22:46:52+00:00
Next Scan 2024-10-06T22:46:52+00:00

Last Scan

Scanned2024-09-29T22:46:52+00:00
URL https://gipsytv.eu/robots.txt
Redirect https://www.gipsytv.eu/robots.txt
Redirect Domain www.gipsytv.eu
Redirect Base gipsytv.eu
Domain IPs 94.130.67.178
Redirect IPs 94.130.67.178
Response IP 94.130.67.178
Found Yes
Hash 7b348e22bb04bd35b045d25ea78f147d75153051ca6be5852b109e2fac3330fe
SimHash 200a5031ef68

Groups

*

Rule Path
Disallow /buxus/admin/
Disallow /buxus/application/
Disallow /buxus/dhtml/
Disallow /buxus/docs/
Disallow /buxus/help/
Disallow /buxus/images/
Disallow /buxus/includes/
Disallow /buxus/lib/
Disallow /buxus/project_includes/
Disallow /buxus/project_scripts/
Disallow /buxus/properties/
Disallow /buxus/scripts/
Disallow /buxus/system/
Disallow /buxus/tmp/

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Information for robots how to crawl the web
  • For development and test server un-comment following line in order to stop crawling test sites
  • Disallow: /
  • For productive Buxus server:
Back to top