violet-path.com
robots.txt

Robots Exclusion Standard data for violet-path.com

Resource Scan

Scan Details

Site Domain violet-path.com
Base Domain violet-path.com
Scan Status Ok
Last Scan2025-07-14T15:40:06+00:00
Next Scan 2025-08-13T15:40:06+00:00

Last Scan

Scanned2025-07-14T15:40:06+00:00
URL https://violet-path.com/robots.txt
Domain IPs 104.21.85.198, 172.67.209.235, 2606:4700:3032::6815:55c6, 2606:4700:3034::ac43:d1eb
Response IP 172.67.209.235
Found Yes
Hash c8c7d9987ab41c54ec9dee39e79b63f744c32d08da6a323f746bcaa91460cc2a
SimHash 500038d0e591

Groups

*

Rule Path
Disallow /thanks.php
Disallow /admin/
Disallow /private/
Disallow /*.php$
Allow /
Allow /images/
Allow /css/
Allow /js/

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Disallow /thanks.php

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Disallow /thanks.php

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Disallow /thanks.php

Other Records

Field Value
crawl-delay 1

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://violet-path.com/sitemap.xml

Comments

  • Crawl-delay for respectful crawling
  • Specific rules for major search engines
  • Block AI training crawlers (optional)
  • Block other unwanted bots