gstv.in
robots.txt

Robots Exclusion Standard data for gstv.in

Resource Scan

Scan Details

Site Domain gstv.in
Base Domain gstv.in
Scan Status Ok
Last Scan2024-06-26T08:10:24+00:00
Next Scan 2024-07-03T08:10:24+00:00

Last Scan

Scanned2024-06-26T08:10:24+00:00
URL https://gstv.in/robots.txt
Redirect https://www.gstv.in/robots.txt
Redirect Domain www.gstv.in
Redirect Base gstv.in
Domain IPs 104.21.39.152, 172.67.146.108, 2606:4700:3033::6815:2798, 2606:4700:3035::ac43:926c
Redirect IPs 104.21.39.152, 172.67.146.108, 2606:4700:3033::6815:2798, 2606:4700:3035::ac43:926c
Response IP 104.21.39.152
Found Yes
Hash a2be92e3bad2f8700af5da981c24f03aa9abf5608917020ee5c8de001136a064
SimHash 88608a306307

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow /backend/

*

Rule Path
Allow /
Allow /backend/public/

Comments

  • Disallow crawling of the backend folder
  • Allow crawling of all other content