news.agentm.tw
robots.txt

Robots Exclusion Standard data for news.agentm.tw

Resource Scan

Scan Details

Site Domain news.agentm.tw
Base Domain agentm.tw
Scan Status Ok
Last Scan2024-11-10T02:04:43+00:00
Next Scan 2024-11-17T02:04:43+00:00

Last Scan

Scanned2024-11-10T02:04:43+00:00
URL https://news.agentm.tw/robots.txt
Domain IPs 104.26.4.161, 104.26.5.161, 172.67.74.107, 2606:4700:20::681a:4a1, 2606:4700:20::681a:5a1, 2606:4700:20::ac43:4a6b
Response IP 104.26.4.161
Found Yes
Hash 6564bf9606e209da14e9a3257227462b52f619ca116a03b4616c3c8540a8b288
SimHash 121f5940b913

Groups

*

Rule Path
Disallow /wp-content/uploads/wpo-plugins-tables-list.json

*

Rule Path
Disallow /wp-json/
Disallow /?rest_route=

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

grapeshot

Rule Path
Disallow

Other Records

Field Value
sitemap http://news.agentm.tw/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK