it.ra.co
robots.txt

Robots Exclusion Standard data for it.ra.co

Resource Scan

Scan Details

Site Domain it.ra.co
Base Domain ra.co
Scan Status Ok
Last Scan2025-06-15T07:29:12+00:00
Next Scan 2025-06-29T07:29:12+00:00

Last Scan

Scanned2025-06-15T07:29:12+00:00
URL https://it.ra.co/robots.txt
Domain IPs 104.22.12.227, 104.22.13.227, 172.67.14.47, 2606:4700:10::6816:ce3, 2606:4700:10::6816:de3, 2606:4700:10::ac43:e2f
Response IP 172.67.14.47
Found Yes
Hash 41fcd392d620465d15905aef0310092c3acb8635201b715cf21e57400229f677
SimHash 797cc816ac12

Groups

*

Rule Path
Disallow /pro$
Disallow /pro/
Disallow /user$
Disallow /user/
Disallow /api$
Disallow /api/
Disallow /my-tickets
Disallow /logout.aspx
Disallow /inbox.aspx
Disallow /widget

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

ias crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

ttd-content

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

verity

Rule Path
Disallow /

admantx

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendiction

Rule Path
Disallow /

applebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.ra.co/sitemap.xml

Comments

  • non-public pages
  • commercial bots
  • ai content scraping bots