ed.ai
robots.txt

Robots Exclusion Standard data for ed.ai

Resource Scan

Scan Details

Site Domain ed.ai
Base Domain ed.ai
Scan Status Ok
Last Scan2025-08-23T07:56:39+00:00
Next Scan 2025-09-22T07:56:39+00:00

Last Scan

Scanned2025-08-23T07:56:39+00:00
URL https://ed.ai/robots.txt
Redirect https://www.ed.ai/robots.txt
Redirect Domain www.ed.ai
Redirect Base ed.ai
Domain IPs 199.60.103.1, 199.60.103.101
Redirect IPs 13.107.246.59
Response IP 13.107.253.59
Found Yes
Hash 54c11b97a6a9665201a8bcc964d54e21081bb33336d51db9bb18afce7ec567b1
SimHash c20219eb24b2

Groups

*

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

ccbot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

claude-web

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.ed.ai/sitemap.xml

Comments

  • Robots.txt for production
  • Sitemap location
  • LLMs information file
  • Specific crawl directives

Warnings

  • `llms.txt` is not a known field.