Log In Sign Up

bskb.com
robots.txt

Robots Exclusion Standard data for bskb.com

Resource Scan

Scan Details

Site Domain bskb.com
Base Domain bskb.com
Scan Status Ok
Last Scan2025-09-04T10:03:40+00:00
Next Scan 2025-10-04T10:03:40+00:00

Last Scan

Scanned2025-09-04T10:03:40+00:00
URL https://bskb.com/robots.txt
Redirect https://www.bskb.com/robots.txt
Redirect Domain www.bskb.com
Redirect Base bskb.com
Domain IPs 107.154.137.3, 45.60.88.3
Redirect IPs 107.154.183.3
Response IP 107.154.183.3
Found Yes
Hash 69cec7be141cabd627b3642dd3f06dbb61ceea3cd8e9db5a309d1d44250d3ff7
SimHash 7819de1347b1

Groups

*

Rule Path
Disallow /

red

Rule Path
Disallow

screaming frog seo spider

Rule Path
Disallow

sitecheck-sitecrawl by siteimprove.com

Rule Path
Disallow

linkcheck by siteimprove.com

Rule Path
Disallow

image size by siteimprove.com

Rule Path
Disallow
Back to top

Comments

  • This file is used to block search engine indexing on all staging URLs
  • Allow www.redbot.org
  • Allow Screaming Frog
  • Allow Siteimprove
Back to top

Warnings

  • `noindex` is not a known field.
Back to top