Log In Sign Up

barmastersmobilebartending.com
robots.txt

Robots Exclusion Standard data for barmastersmobilebartending.com

Resource Scan

Scan Details

Site Domain barmastersmobilebartending.com
Base Domain barmastersmobilebartending.com
Scan Status Ok
Last Scan2025-12-23T06:09:29+00:00
Next Scan 2026-01-22T06:09:29+00:00

Last Scan

Scanned2025-12-23T06:09:29+00:00
URL https://barmastersmobilebartending.com/robots.txt
Domain IPs 38.88.82.64
Response IP 38.88.82.64
Found Yes
Hash 02b9a47d6b46c29ca6bf92987470dba6c929f6f9389691b918d06e4a5d31115b
SimHash 5850d990c457

Groups

*

Rule Path
Allow /
Allow /wp-content/plugins/google-sitemap-generator
Disallow /wp-admin
Disallow /wp-login
Disallow /showcase
Disallow /thank-you

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

claude-web

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

youbot

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://barnatcrescentlake.com/sitemap.xml
Back to top

Comments

  • --- AI Crawlers ---
  • Block GPTBot (used for model training)
  • Allow ChatGPT link preview fetcher (needed for real-time browsing only)
  • Anthropic (Claude AI)
  • Perplexity
  • You.com
  • Apple generative AI (Applebot-Extended)
  • CCBot (Cohere)
Back to top