static.thairath.co.th
robots.txt

Robots Exclusion Standard data for static.thairath.co.th

Resource Scan

Scan Details

Site Domain static.thairath.co.th
Base Domain thairath.co.th
Scan Status Ok
Last Scan2025-02-19T09:16:55+00:00
Next Scan 2025-03-21T09:16:55+00:00

Last Scan

Scanned2025-02-19T09:16:55+00:00
URL https://static.thairath.co.th/robots.txt
Domain IPs 156.227.14.33, 156.227.14.34, 38.60.148.98, 38.60.148.99
Response IP 156.227.14.33
Found Yes
Hash 5ffbd9016e0341db48e9ffe42aa4ec790843ab9660f4842f432b49aa7727817c
SimHash 40101d57ebb7

Groups

*

Rule Path
Disallow

grapeshot

Rule Path
Disallow

ias_crawler

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.thairath.co.th/sitemap.xml

Comments

  • Allow all user agents to access everything
  • Explicitly allow certain crawlers (grapeshot and ias_crawler) to access everything
  • Provide a reference to your sitemap