Log In Sign Up

sponge.co.uk
robots.txt

Robots Exclusion Standard data for sponge.co.uk

Resource Scan

Scan Details

Site Domain sponge.co.uk
Base Domain sponge.co.uk
Scan Status Ok
Last Scan2026-01-24T00:40:32+00:00
Next Scan 2026-02-23T00:40:32+00:00

Last Scan

Scanned2026-01-24T00:40:32+00:00
URL https://sponge.co.uk/robots.txt
Redirect https://www.sponge.co.uk/robots.txt
Redirect Domain www.sponge.co.uk
Redirect Base sponge.co.uk
Domain IPs 104.26.6.40, 104.26.7.40, 172.67.74.121, 2606:4700:20::681a:628, 2606:4700:20::681a:728, 2606:4700:20::ac43:4a79
Redirect IPs 104.26.6.40, 104.26.7.40, 172.67.74.121, 2606:4700:20::681a:628, 2606:4700:20::681a:728, 2606:4700:20::ac43:4a79
Response IP 104.26.6.40
Found Yes
Hash 51e160fc9a6d8fb8ac990d02671119ed1b2cccf6c302b29e036808be6237b3e5
SimHash 6415f0e2c393

Groups

*

Rule Path
Disallow /api/
Disallow /flow/
Disallow /basket
Disallow /basket/
Disallow /checkout/
Disallow /account
Disallow /account/
Disallow /cake-card/start
Disallow /blog/main-site-content
Disallow /blog/main-site-content/
Disallow /blog/app_preview/
Disallow /blog/app_preview
Disallow /blog/b2b_preview/
Disallow /blog/b2b_preview
Disallow /ab/
Disallow /sponge-friday/addtobasket
Disallow /sponge-friday/addtobasket/
Disallow /sponge-of-the-week/addtobasket
Disallow /sponge-of-the-week/addtobasket/
Disallow /blog/index.php/
Disallow /index.php/
Back to top

Other Records

Field Value
sitemap https://www.sponge.co.uk/sitemap.xml
sitemap https://www.sponge.co.uk/blog/sitemap.xml
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
Back to top