usopen.org
robots.txt

Robots Exclusion Standard data for usopen.org

Resource Scan

Scan Details

Site Domain usopen.org
Base Domain usopen.org
Scan Status Ok
Last Scan2024-09-17T19:28:40+00:00
Next Scan 2024-09-24T19:28:40+00:00

Last Scan

Scanned2024-09-17T19:28:40+00:00
URL https://usopen.org/robots.txt
Redirect https://www.usopen.org/robots.txt
Redirect Domain www.usopen.org
Redirect Base usopen.org
Domain IPs 125.252.228.139, 2600:1413:b000:683::26a4, 2600:1413:b000:69b::26a4
Redirect IPs 125.252.228.139, 2600:1413:b000:683::26a4, 2600:1413:b000:69b::26a4
Response IP 104.69.41.67
Found Yes
Hash 94dc7a5cb8ff912066c9015fabc967f4dddf4b300a7c47a1234842748dbb81a7
SimHash c8195b201747

Groups

twitterbot

Rule Path
Allow /images

googlebot

Rule Path
Allow /
Allow /assets
Allow /uso/css
Allow /uso/js
Allow /images

*

Rule Path
Disallow /p/
Disallow /s/
Disallow /cgi-bin/
Disallow /rc/
Disallow /slsearch
Disallow /demos
Disallow /uso
Disallow /ios
Disallow /pdf/private
Disallow /en_US/xml
Disallow /en_US/includes
Disallow /en_US/slamtracker/slamtracker.html
Disallow /en_US/feedback
Disallow /en_US/scores/stats
Disallow /en_US/scores/xml
Disallow /en_US/event_guide
Disallow /zh_CN
Disallow /ecp
Disallow /webview

Comments

  • robots.txt
  • Format is:
  • User-agent: <name of spider>
  • Disallow: <nothing> | <path>
  • -----------------------------------------------------------------------------