rswww.com
robots.txt

Robots Exclusion Standard data for rswww.com

Resource Scan

Scan Details

Site Domain rswww.com
Base Domain rswww.com
Scan Status Ok
Last Scan2025-11-13T21:33:40+00:00
Next Scan 2025-12-13T21:33:40+00:00

Last Scan

Scanned2025-11-13T21:33:40+00:00
URL https://rswww.com/robots.txt
Redirect https://uk.rs-online.com/robots.txt
Redirect Domain uk.rs-online.com
Redirect Base rs-online.com
Domain IPs 35.71.139.74, 52.223.17.4
Redirect IPs 23.15.103.36
Response IP 118.215.83.37
Found Yes
Hash a1bc8666c95c00783ee24a209c002df04d8fcd28d629bcfddfbe518ab42d763e
SimHash 79552b19f620

Groups

oai-searchbot
chatgpt-user
perplexitybot
perplexity-user
claude-user
claude-searchbot

Rule Path
Allow /

*

Rule Path
Disallow /offline
Disallow /web/cart/
Disallow /web/orderoptions/
Disallow /web/register/registration
Disallow /web/feedback.html
Disallow /web/orderhistory.html
Disallow /cgi-bin/
Disallow /web/ca/
Disallow /web/ma/
Disallow /web/cp*
Disallow /web/template/
Disallow /web/docs/
Disallow /web/auth/
Disallow /web/lc/
Disallow /web/miniBasketRunningTotal.html
Disallow /web/services/header/getDynamicData
Disallow /pm/*
Disallow /myaccount/
Disallow /basket
Disallow /*login
Disallow /*%26cid*
Disallow /*?viewMode=*
Disallow /*?sra=*
Disallow /*%26rpp%3D*
Disallow /*?Nr*
Disallow /*?mpn=*
Disallow /*?esid=*
Disallow /*?brandShopPath=*
Disallow /*lastAttributeSelectedBlock%3D*
Disallow /*%26vn%3D*
Disallow /*?vn=*
Disallow /*?N=*
Disallow /*?BV_SessionID=*
Disallow /web/web/*
Disallow /*?page-offset=*
Disallow /*faq/faq*
Disallow /*?method=*
Disallow /*%26method%3D*
Disallow /*%26sort-by%3D*
Disallow *searchTerm%3D*
Disallow /mobile/basket*
Disallow /mobile/emptyBasket*
Disallow /mobile/co/
Disallow /mobile/services/
Disallow /mobile/guestLogin*
Disallow /mobile/register/
Disallow /mobile/secure/

Comments

  • Allow Agentic Search
  • mobile environment