capitalhairandbeauty.co.uk
robots.txt

Robots Exclusion Standard data for capitalhairandbeauty.co.uk

Resource Scan

Scan Details

Site Domain capitalhairandbeauty.co.uk
Base Domain capitalhairandbeauty.co.uk
Scan Status Ok
Last Scan2024-09-20T12:12:46+00:00
Next Scan 2024-10-20T12:12:46+00:00

Last Scan

Scanned2024-09-20T12:12:46+00:00
URL https://capitalhairandbeauty.co.uk/robots.txt
Redirect https://www.capitalhairandbeauty.co.uk/robots.txt
Redirect Domain www.capitalhairandbeauty.co.uk
Redirect Base capitalhairandbeauty.co.uk
Domain IPs 93.184.255.33
Redirect IPs 93.184.255.33
Response IP 93.184.255.33
Found Yes
Hash 4928ae08aa7f9d60c9bc9ad2508f3c93705893ab34f72d83052e49ae452d6fa4
SimHash 265051a4c792

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yeti

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

baiduspider-image

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

sogou spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp china

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

ahrefsbot
baiduspider-video
baiduspider-news
baiduspider-favo
baiduspider-cpro
baiduspider-ads
mj12bot
blexbot
ezooms
mozilla/4.0 (compatible; msie 6.0; windows nt 7.0) xspider 7
aspiegelbot

Rule Path
Disallow /

*

Rule Path
Disallow /*Basket.aspx
Disallow /*Checkout.aspx
Disallow /*Login.aspx
Disallow /*checkout
Disallow /*login
Disallow /*basket
Disallow /*?*filter=*
Disallow /*?*donotindex=*
Disallow /*?*sortorder=*
Disallow /*?*listing_pagesize=*
Disallow /students
Disallow /*?*searchterm=*
Disallow /*?*expand=*
Disallow /leeds-only*

Other Records

Field Value
sitemap https://www.capitalhairandbeauty.co.uk/sitemap_index.xml

Comments

  • Bing
  • Yeti (Naver)
  • Baidu
  • SoGou
  • Yahoo! Slurp China
  • Yandex