Log In Sign Up

ori.net
robots.txt

Robots Exclusion Standard data for ori.net

Resource Scan

Scan Details

Site Domain ori.net
Base Domain ori.net
Scan Status Ok
Last Scan2026-02-02T23:09:56+00:00
Next Scan 2026-03-04T23:09:56+00:00

Last Scan

Scanned2026-02-02T23:09:56+00:00
URL https://ori.net/robots.txt
Domain IPs 104.21.37.164, 172.67.210.148, 2606:4700:3034::ac43:d294, 2606:4700:3035::6815:25a4
Response IP 172.67.210.148
Found Yes
Hash ee639c343ff77f3fb19e26f31f17ce9e8ff24eed3ece8b8e48ea7c3b638f4968
SimHash 701fee536d92

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://ori.net/sitemap-index.xml
Back to top

Comments

  • Robots.txt for ORI.NET
  • https://ori.net
  • Allow all crawlers
  • Disallow admin or private paths (if any exist in the future)
  • Disallow: /admin/
  • Disallow: /api/
  • Sitemap location
  • Crawl-delay for respectful crawling (optional)
  • Crawl-delay: 1
  • Specific rules for major search engines
  • Google
  • Bing
  • Yahoo/Slurp
  • Block bad bots (optional - add as needed)
Back to top