Log In Sign Up

skins.ow2.org
robots.txt

Robots Exclusion Standard data for skins.ow2.org

Resource Scan

Scan Details

Site Domain skins.ow2.org
Base Domain ow2.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-06-24T00:17:48+00:00
Next Scan 2025-09-22T00:17:48+00:00

Last Successful Scan

Scanned2024-11-04T00:15:59+00:00
URL https://skins.ow2.org/robots.txt
Redirect https://www.ow2.org/download/Main/WebHome/robots.txt
Redirect Domain www.ow2.org
Redirect Base ow2.org
Domain IPs 81.200.36.119
Redirect IPs 54.38.13.219
Response IP 54.38.13.219
Found Yes
Hash 5cadf3bb41f1f91db96d7eb7b502ce2e2a6c13d0b97ecc0eeec184c8d5bddad7
SimHash b428c9f58ea5

Groups

semrushbot
siteauditbot
ahrefsbot
coherencebot
deskyobot
magpie-crawler
mauibot
coccocbot-image
coccocbot-web
dotbot
infotigerbot
mail.ru_bot
mj12bot
seznambot
surdotlybot
wellknownbot
yandexbot
dataforseobot
sabsimbot
trendictionbot
yacybot
zoominfobot

Rule Path
Disallow /

*

Rule Path
Disallow /view/services/
Disallow /view/Membership_Joining/On_Line_Registration
Disallow /services/
Disallow /status/
Disallow /xmlrpc/
Disallow /view/XWiki

*

Rule Path
Disallow /viewattachrev/
Disallow /viewrev/
Disallow /pdf/
Disallow /tex/
Disallow /edit/
Disallow /create/
Disallow /inline/
Disallow /preview/
Disallow /save/
Disallow /saveandcontinue/
Disallow /rollback/
Disallow /deleteversions/
Disallow /cancel/
Disallow /delete/
Disallow /deletespace/
Disallow /undelete/
Disallow /reset/
Disallow /register/
Disallow /propupdate/
Disallow /propadd/
Disallow /propdisable/
Disallow /propenable/
Disallow /propdelete/
Disallow /objectadd/
Disallow /commentadd/
Disallow /commentsave/
Disallow /objectsync/
Disallow /objectremove/
Disallow /attach/
Disallow /upload/
Disallow /download/
Disallow /temp/
Disallow /downloadrev/
Disallow /dot/
Disallow /svg/
Disallow /delattachment/
Disallow /login/
Disallow /loginsubmit/
Disallow /loginerror/
Disallow /logout/
Disallow /charting/
Disallow /lock/
Disallow /redirect/
Disallow /admin/
Disallow /export/
Disallow /import/
Disallow /get/
Disallow /distribution/
Disallow /imagecaptcha/
Disallow /unknown/
Disallow /view/Sandbox/
Disallow /view/Admin/
Disallow /view/Stats/
Disallow /view/Panels/
Disallow /Main/Search
Disallow /xwiki/rest/
Back to top

Comments

  • Disallow all the website to undesirable bots
  • Syntax reference: https://developers.google.com/search/docs/advanced/robots/create-robots-txt
  • OW2 Custom
  • XWIKI recommendations
  • https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Performances/#HRobots.txt
  • Prevent bots from executing all actions except "view" since:
  • 1) we don't want bots to execute stuff in the wiki!
  • 2) we don't want bots to consume CPU and memory
  • (for example to perform exports)
  • Don't index sandbox content since it's sample content
  • Don't index Admin space since it contains Admin stuff.
  • Note that the Admin space is protected by permissions
  • anyway but this acts as a safety net to not have private
  • info leaked on the internet ;)
  • Don't index Stats data (just because it's not useful and
  • those pages are a bit CPU intensive)
  • Don't index Panels data (because we don't want it
  • indexed on the internet)
  • Don't index the search page.
  • Don't index the REST API.
  • These are just UI elements which can cause infinite loops in
  • web crawlers. See https://jira.xwiki.org/browse/XWIKI-16915
Back to top

Warnings

  • 1 invalid line.
Back to top