Log In Sign Up

beecrownlogistics.co.uk
robots.txt

Robots Exclusion Standard data for beecrownlogistics.co.uk

Resource Scan

Scan Details

Site Domain beecrownlogistics.co.uk
Base Domain beecrownlogistics.co.uk
Scan Status Ok
Last Scan2026-02-14T10:18:48+00:00
Next Scan 2026-02-21T10:18:48+00:00

Last Scan

Scanned2026-02-14T10:18:48+00:00
URL https://beecrownlogistics.co.uk/robots.txt
Domain IPs 88.99.99.104
Response IP 88.99.99.104
Found Yes
Hash 20a62d9725f6060476092f4315c59e32a9e5ab7a2da37c02cddb48aa00a39b33
SimHash 7a10584085e3

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-admin/admin-ajax.php
Allow /

Other Records

Field Value
crawl-delay 10

oai-searchbot

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

gemini

Rule Path
Allow /

claudebot

Rule Path
Allow /

ccbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckassistbot

Rule Path
Allow /

openai

Rule Path
Allow /

generativeai

Rule Path
Allow /

predictiveai

Rule Path
Allow /

tiktokbot

Rule Path
Allow /

facebot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

instagram

Rule Path
Allow /

applebot

Rule Path
Allow /

alexacrawler

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://beecrownlogistics.com/wp-sitemap.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • Allow AI and generative bots
  • Allow social media crawlers
  • Allow Apple
Back to top