logic.al
robots.txt

Robots Exclusion Standard data for logic.al

Resource Scan

Scan Details

Site Domain logic.al
Base Domain logic.al
Scan Status Ok
Last Scan2025-09-04T14:35:11+00:00
Next Scan 2025-10-04T14:35:11+00:00

Last Scan

Scanned2025-09-04T14:35:11+00:00
URL https://logic.al/robots.txt
Domain IPs 104.26.0.180, 104.26.1.180, 172.67.72.169, 2606:4700:20::681a:1b4, 2606:4700:20::681a:b4, 2606:4700:20::ac43:48a9
Response IP 104.26.1.180
Found Yes
Hash 6889c046dba1be7ff6bf59549df394c3342510fd264451a829349cfb411ea0d8
SimHash 6417cc71e71a

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /cart/
Disallow /checkout/
Disallow /my-account/
Disallow /xmlrpc.php
Disallow /*?s=*

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

openai

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

Comments

  • Allow major search engines with crawl-delay
  • Block known bad bots and scrapers
  • Block chatbots scraping for training
  • Optional: Prevent image indexing (remove if you want image SEO)