Log In Sign Up

chilternseeds.co.uk
robots.txt

Robots Exclusion Standard data for chilternseeds.co.uk

Resource Scan

Scan Details

Site Domain chilternseeds.co.uk
Base Domain chilternseeds.co.uk
Scan Status Ok
Last Scan2024-09-22T10:32:57+00:00
Next Scan 2024-10-22T10:32:57+00:00

Last Scan

Scanned2024-09-22T10:32:57+00:00
URL https://chilternseeds.co.uk/robots.txt
Redirect https://www.chilternseeds.co.uk/robots.txt
Redirect Domain www.chilternseeds.co.uk
Redirect Base chilternseeds.co.uk
Domain IPs 104.26.4.59, 104.26.5.59, 172.67.71.135, 2606:4700:20::681a:43b, 2606:4700:20::681a:53b, 2606:4700:20::ac43:4787
Redirect IPs 104.26.4.59, 104.26.5.59, 172.67.71.135, 2606:4700:20::681a:43b, 2606:4700:20::681a:53b, 2606:4700:20::ac43:4787
Response IP 104.26.5.59
Found Yes
Hash 34eee1768a50dcdff5187ae2305e017d2f53d55c5634efb4a273e35b7674157d
SimHash 2d11d1d156b1

Groups

architextspider
baiduspider
googlebot
googlebot-mobile
adsbot-google
googlebot-image
mediapartners-google
msnbot
bingbot
msnbot-media
scooter
semrushbot-sa
slurp
teoma
yahoo-blogs
yahoo-mmcrawler
yandex

Rule Path
Allow /
Disallow /khaos/*
Disallow /basket.php
Disallow /checkout*
Disallow /ajax/
Disallow /search*

sosospider

Rule Path
Disallow /

*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.chilternseeds.co.uk/xml_sitemap.php
Back to top

Comments

  • allow only important bots
Back to top

Warnings

  • 1 invalid line.
Back to top