weblog.viet.net
robots.txt

Robots Exclusion Standard data for weblog.viet.net

Resource Scan

Scan Details

Site Domain weblog.viet.net
Base Domain viet.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-08-05T09:24:40+00:00
Next Scan 2025-11-03T09:24:40+00:00

Last Successful Scan

Scanned2025-03-16T09:21:56+00:00
URL https://weblog.viet.net/robots.txt
Domain IPs 104.21.62.138, 172.67.168.90, 2606:4700:3032::6815:3e8a, 2606:4700:3035::ac43:a85a
Response IP 104.21.62.138
Found Yes
Hash 6fc96e645384f7fd5cab8f109d9c1aecbd3bee5664b51d8b23a071ce77ae64b9
SimHash b9107265cf11

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /pipermail/

Comments

  • Exclude robots from the cgi areas