thevictorianemporium.com
robots.txt

Robots Exclusion Standard data for thevictorianemporium.com

Resource Scan

Scan Details

Site Domain thevictorianemporium.com
Base Domain thevictorianemporium.com
Scan Status Ok
Last Scan2024-08-22T06:44:14+00:00
Next Scan 2024-09-21T06:44:14+00:00

Last Scan

Scanned2024-08-22T06:44:14+00:00
URL https://thevictorianemporium.com/robots.txt
Redirect https://www.thevictorianemporium.com/robots.txt
Redirect Domain www.thevictorianemporium.com
Redirect Base thevictorianemporium.com
Domain IPs 104.26.14.222, 104.26.15.222, 172.67.69.112, 2606:4700:20::681a:ede, 2606:4700:20::681a:fde, 2606:4700:20::ac43:4570
Redirect IPs 104.26.14.222, 104.26.15.222, 172.67.69.112, 2606:4700:20::681a:ede, 2606:4700:20::681a:fde, 2606:4700:20::ac43:4570
Response IP 104.26.14.222
Found Yes
Hash ceffbf8c724aedbeb14e8d61385eafa94bc3788d82afa226cc0b2d2d8fa26aeb
SimHash 1e9859c7488e

Groups

*

Rule Path
Disallow /includes/
Disallow /store_includes/
Disallow /plugins/feedback.php
Disallow /antiques/contact/
Disallow /account/
Disallow /wishlist
Disallow /store/basket
Disallow /checkout
Disallow /search/
Disallow /*history/category/
Disallow /*tips/category/
Disallow /*advice/category/
Disallow /*editorial/category/
Disallow /publications/tag/
Disallow /?ACT=
Disallow /?URL=
Disallow /*?filter

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.thevictorianemporium.com/sitemap.xml

Comments

  • robots.txt for http://www.thevictorianemporium.com/
  • Disallow: /*P2
  • Disallow: /*P3
  • Disallow: /*P4
  • Disallow: /*P5
  • Disallow: /*P6
  • Disallow: /*P7
  • Disallow: /*P8
  • Disallow: /*P9