yourarticlestore.com
robots.txt

Robots Exclusion Standard data for yourarticlestore.com

Resource Scan

Scan Details

Site Domain yourarticlestore.com
Base Domain yourarticlestore.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2025-11-11T14:14:27+00:00
Next Scan 2025-12-11T14:14:27+00:00

Last Successful Scan

Scanned2025-10-12T21:32:46+00:00
URL https://yourarticlestore.com/robots.txt
Domain IPs 146.59.68.41
Response IP 146.59.68.41
Found Yes
Hash 75834c4f6b2ba45835137b902599ef0493774a8f2b45be47f08f918bf0bb4f9a
SimHash 221e5a11e5f2

Groups

*

Rule Path
Allow /
Allow /index.html
Allow /contact.html
Allow /login.html
Allow /assets/
Allow /assets/images/
Allow /assets/*.svg
Allow /assets/images/*.webp
Allow /manifest.json
Disallow /admin/
Disallow /private/
Disallow /.git/
Disallow /.env
Disallow /config/
Disallow /database/

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

badbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

telegrambot

Rule Path
Allow /

Other Records

Field Value
sitemap https://yourarticlestore.com/sitemap.xml

Comments

  • robots.txt for หมูบิน168 - สล็อตเว็บตรงไม่ผ่านเอเย่นต์
  • Allow all search engines to index the site
  • Block access to sensitive files
  • Sitemap location
  • Crawl-delay for polite crawling (1 second)
  • Specific rules for major search engines
  • Block malicious bots
  • Allow social media crawlers