stocksy.com
robots.txt

Robots Exclusion Standard data for stocksy.com

Resource Scan

Scan Details

Site Domain stocksy.com
Base Domain stocksy.com
Scan Status Ok
Last Scan2024-11-13T01:15:01+00:00
Next Scan 2024-12-13T01:15:01+00:00

Last Scan

Scanned2024-11-13T01:15:01+00:00
URL https://stocksy.com/robots.txt
Redirect https://www.stocksy.com/robots.txt
Redirect Domain www.stocksy.com
Redirect Base stocksy.com
Domain IPs 104.17.2.117, 104.17.3.117, 2606:4700::6811:275, 2606:4700::6811:375
Redirect IPs 104.17.2.117, 104.17.3.117, 2606:4700::6811:275, 2606:4700::6811:375
Response IP 104.17.2.117
Found Yes
Hash 0409d88eae76a1e454648fbdd2757e502fd354af947127178de6934c79f535ea
SimHash e915dc52e762

Groups

*

Rule Path
Allow /user/login$
Allow /user/signup$
Disallow /*/user/signup
Disallow /*/user/login
Disallow /user/signup?
Disallow /user/login?
Disallow /user/login?referer=*
Disallow /user/signup?referer=*
Disallow /cart/complete
Disallow /user/lostpassword
Disallow /es/user/lostpassword
Disallow /user/logout
Disallow /es/user/logout
Disallow /user/edit-profile
Disallow /es/user/edit-profile
Disallow /embed/video*
Disallow /modelssearch*
Disallow /es/modelssearch*
Disallow /search*
Disallow /es/search*

semrushbot-sa

Rule Path
Disallow /user/login
Disallow /user/signup

semrushbot-ba

Rule Path
Disallow /user/login
Disallow /user/signup

semrushbot-si

Rule Path
Disallow /user/login
Disallow /user/signup

semrushbot-swa

Rule Path
Disallow /user/login
Disallow /user/signup

semrushbot-ct

Rule Path
Disallow /user/login
Disallow /user/signup

semrushbot-bm

Rule Path
Disallow /user/login
Disallow /user/signup

splitsignalbot

Rule Path
Disallow /user/login
Disallow /user/signup

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://sitemaps.stocksy.com/sitemap-september-2024.txt

Warnings

  • `noindex` is not a known field.