Log In Sign Up

portal.ieu-monitoring.com
robots.txt

Robots Exclusion Standard data for portal.ieu-monitoring.com

Resource Scan

Scan Details

Site Domain portal.ieu-monitoring.com
Base Domain ieu-monitoring.com
Scan Status Ok
Last Scan2025-10-01T07:04:09+00:00
Next Scan 2025-10-02T07:04:09+00:00

Last Scan

Scanned2025-10-01T07:04:09+00:00
URL https://portal.ieu-monitoring.com/robots.txt
Redirect https://ieu-monitoring.com/robots.txt
Redirect Domain ieu-monitoring.com
Redirect Base ieu-monitoring.com
Domain IPs 185.175.196.10, 2a00:1c98:60:1020::1f8d:e936
Redirect IPs 185.175.196.10, 2a00:1c98:60:1020::1f8d:e936
Response IP 185.175.196.10
Found Yes
Hash b97fdb4a8ba21f3ade9d13c3fe0fdb82b0ad43c98dc88f827e246ef32d3270ff
SimHash 42381951c787

Groups

googlebot

Rule Path
Disallow /wp-admin/

bingbot

Rule Path
Disallow /wp-admin/

slurp

Rule Path
Disallow /wp-admin/

baiduspider

Rule Path
Disallow /wp-admin/

yandex

Rule Path
Disallow /wp-admin/

*

Rule Path
Disallow /wp-admin/

gptbot

Rule Path
Disallow /private/
Allow /

google-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://ieu-monitoring.com/ieu-sitemap.xml
Back to top

Comments

  • Allow search engine bots except in /wp-admin/
  • General rule for all other compliant bots
  • Block download and training by AI crawlers, allow basic indexing and citation
  • GPTBot: browsing allowed, no training without license
  • GPTBot may index and cite for browsing purposes only
  • Commercial AI training only allowed via license: https://ieu-monitoring.com/ieu-store/Insight-EU-GPT-AI-Annual-Subscription-p711540131
  • Block other AI crawlers completely unless licensed
  • AI usage policy notice
  • The content of this website may not be downloaded, stored, or used for AI model training
  • without a prior licensing agreement.
  • See: https://ieu-monitoring.com/ieu-store/Insight-EU-GPT-AI-Annual-Subscription-p711540131.
Back to top