Log In Sign Up

rckeith.co.uk
robots.txt

Robots Exclusion Standard data for rckeith.co.uk

Resource Scan

Scan Details

Site Domain rckeith.co.uk
Base Domain rckeith.co.uk
Scan Status Ok
Last Scan2025-09-25T20:15:51+00:00
Next Scan 2025-10-02T20:15:51+00:00

Last Scan

Scanned2025-09-25T20:15:51+00:00
URL https://rckeith.co.uk/robots.txt
Domain IPs 185.151.30.169, 2a07:7800::169
Response IP 185.151.30.169
Found Yes
Hash bed6c770a6503d6942d0c16f54d240a1444798d87514dc2e0c37f9a5d301eff3
SimHash 6810ba00e597

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://rckeith.co.uk/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • Allow Google, Bing, and major search engines
  • Allow OpenAI GPTBot (used by ChatGPT)
  • Allow ChatGPT-User agent (used for browsing)
  • Disallow all other bots by default
  • ---------------------------
  • END YOAST BLOCK
Back to top