cgi.widen.net
robots.txt

Robots Exclusion Standard data for cgi.widen.net

Resource Scan

Scan Details

Site Domain cgi.widen.net
Base Domain widen.net
Scan Status Ok
Last Scan2026-01-21T05:45:58+00:00
Next Scan 2026-02-20T05:45:58+00:00

Last Scan

Scanned2026-01-21T05:45:58+00:00
URL https://cgi.widen.net/robots.txt
Domain IPs 65.8.76.39, 65.8.76.41, 65.8.76.74, 65.8.76.81
Response IP 65.8.76.81
Found Yes
Hash be1733ac447528238c415708ba201513acb5a494c435efebc62924c7f7e8149d
SimHash c844cb16cfb1

Groups

twitterbot

Rule Path
Disallow

*

Rule Path
Allow /content/
Allow /view/
Disallow /

Comments

  • Twitter cards require Twitterbot access to all paths