snet.gob.sv
robots.txt

Robots Exclusion Standard data for snet.gob.sv

Resource Scan

Scan Details

Site Domain snet.gob.sv
Base Domain snet.gob.sv
Scan Status Ok
Last Scan2025-05-09T21:54:14+00:00
Next Scan 2025-06-08T21:54:14+00:00

Last Scan

Scanned2025-05-09T21:54:14+00:00
URL https://www.snet.gob.sv/robots.txt
Domain IPs 190.151.176.101
Response IP 190.151.176.101
Found Yes
Hash dd0e5632ae47bcec23a80c5e7d3fd63c451c885c051470d6ce5dc608339b04f8
SimHash b18448508ae6

Groups

*

Rule Path
Disallow /eliseo/
Disallow /googlemaps/
Disallow /admin/
Disallow /Icons/
Disallow /virtualearth/
Disallow /soporte/

susedig

Rule Path
Disallow

stress-agent

Rule Path
Disallow /

meta-externalagent

Rule Path Comment
Disallow / Allow everything

Comments

  • exclude help system from robots
  • but allow htdig to index our doc-tree
  • disallow stress test
  • Disallow a specific directory